You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/23 13:55:23 UTC
[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0-ea-b93) - Build #
6199 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6199/
Java: 64bit/jdk1.8.0-ea-b93 -XX:+UseCompressedOops -XX:+UseG1GC
1 tests failed.
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:34080/collection1lastClient and got 243 from http://127.0.0.1:58505/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:34080/collection1lastClient and got 243 from http://127.0.0.1:58505/collection1
at __randomizedtesting.SeedInfo.seed([7C221E7B74E4A52:8624AFFFC0112A6E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:491)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 9722 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 544740 T3032 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 544744 T3032 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1371987938478
[junit4:junit4] 2> 544745 T3032 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 544745 T3033 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 544846 T3032 oasc.ZkTestServer.run start zk server on port:33984
[junit4:junit4] 2> 544846 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 544917 T3039 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3e969ade name:ZooKeeperConnection Watcher:127.0.0.1:33984 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 544917 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 544918 T3032 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 544924 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 544927 T3041 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3051637c name:ZooKeeperConnection Watcher:127.0.0.1:33984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 544928 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 544928 T3032 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 544932 T3032 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 544936 T3032 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 544938 T3032 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 544942 T3032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 544943 T3032 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 544949 T3032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 544949 T3032 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 544954 T3032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 544955 T3032 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 544958 T3032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 544959 T3032 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 544962 T3032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 544963 T3032 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 544966 T3032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 544967 T3032 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 544970 T3032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 544971 T3032 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 544974 T3032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 544975 T3032 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 544978 T3032 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 544979 T3032 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 545061 T3032 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 545064 T3032 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:38912
[junit4:junit4] 2> 545064 T3032 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 545065 T3032 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 545065 T3032 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371987938716
[junit4:junit4] 2> 545065 T3032 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371987938716/solr.xml
[junit4:junit4] 2> 545065 T3032 oasc.CoreContainer.<init> New CoreContainer 66580832
[junit4:junit4] 2> 545066 T3032 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371987938716/'
[junit4:junit4] 2> 545066 T3032 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371987938716/'
[junit4:junit4] 2> 545106 T3032 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 545106 T3032 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 545107 T3032 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 545107 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 545107 T3032 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 545107 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 545108 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 545108 T3032 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 545108 T3032 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 545109 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 545112 T3032 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 545112 T3032 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33984/solr
[junit4:junit4] 2> 545112 T3032 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 545113 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 545114 T3052 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6aa72ab1 name:ZooKeeperConnection Watcher:127.0.0.1:33984 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 545115 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 545116 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 545118 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 545120 T3054 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49ab338a name:ZooKeeperConnection Watcher:127.0.0.1:33984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 545120 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 545121 T3032 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 545124 T3032 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 545126 T3032 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 545127 T3032 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38912_
[junit4:junit4] 2> 545128 T3032 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38912_
[junit4:junit4] 2> 545132 T3032 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 545137 T3032 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 545139 T3032 oasc.Overseer.start Overseer (id=89914601536225283-127.0.0.1:38912_-n_0000000000) starting
[junit4:junit4] 2> 545143 T3032 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 545146 T3056 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 545147 T3032 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 545149 T3032 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 545151 T3032 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 545155 T3055 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 545158 T3057 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 545158 T3057 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 546658 T3055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 546659 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:38912_",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:38912"}
[junit4:junit4] 2> 546659 T3055 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 546659 T3055 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 546663 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 547160 T3057 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371987938716/collection1
[junit4:junit4] 2> 547160 T3057 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 547162 T3057 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 547162 T3057 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 547164 T3057 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371987938716/collection1/'
[junit4:junit4] 2> 547165 T3057 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371987938716/collection1/lib/README' to classloader
[junit4:junit4] 2> 547165 T3057 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371987938716/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 547190 T3057 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 547225 T3057 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 547226 T3057 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 547229 T3057 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 547540 T3057 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 547540 T3057 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 547541 T3057 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 547545 T3057 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 547548 T3057 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 547558 T3057 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 547561 T3057 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 547563 T3057 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 547564 T3057 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 547564 T3057 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 547564 T3057 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 547565 T3057 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 547565 T3057 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 547565 T3057 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 547566 T3057 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371987938716/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data/
[junit4:junit4] 2> 547566 T3057 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4afb9929
[junit4:junit4] 2> 547566 T3057 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 547566 T3057 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data
[junit4:junit4] 2> 547567 T3057 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data/index/
[junit4:junit4] 2> 547567 T3057 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 547567 T3057 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data/index
[junit4:junit4] 2> 547571 T3057 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2adb1318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 547571 T3057 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 547574 T3057 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 547574 T3057 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 547575 T3057 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 547575 T3057 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 547576 T3057 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 547576 T3057 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 547576 T3057 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 547577 T3057 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 547577 T3057 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 547579 T3057 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 547584 T3057 oass.SolrIndexSearcher.<init> Opening Searcher@28e17612 main
[junit4:junit4] 2> 547585 T3057 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 547585 T3057 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 547590 T3058 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@28e17612 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 547591 T3057 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 547591 T3057 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38912 collection:control_collection shard:shard1
[junit4:junit4] 2> 547593 T3057 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 547600 T3057 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 547603 T3057 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 547604 T3057 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 547604 T3057 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:38912/collection1/
[junit4:junit4] 2> 547604 T3057 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 547604 T3057 oasc.SyncStrategy.syncToMe http://127.0.0.1:38912/collection1/ has no replicas
[junit4:junit4] 2> 547604 T3057 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:38912/collection1/
[junit4:junit4] 2> 547604 T3057 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 548167 T3055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 548174 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 548211 T3057 oasc.ZkController.register We are http://127.0.0.1:38912/collection1/ and leader is http://127.0.0.1:38912/collection1/
[junit4:junit4] 2> 548212 T3057 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38912
[junit4:junit4] 2> 548212 T3057 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 548212 T3057 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 548212 T3057 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 548213 T3057 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 548214 T3032 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 548214 T3032 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 548215 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 548218 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 548218 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 548220 T3061 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@645fd43 name:ZooKeeperConnection Watcher:127.0.0.1:33984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 548220 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 548221 T3032 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 548222 T3032 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 548282 T3032 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 548284 T3032 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34080
[junit4:junit4] 2> 548284 T3032 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 548284 T3032 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 548285 T3032 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371987941956
[junit4:junit4] 2> 548285 T3032 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371987941956/solr.xml
[junit4:junit4] 2> 548285 T3032 oasc.CoreContainer.<init> New CoreContainer 379711305
[junit4:junit4] 2> 548285 T3032 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371987941956/'
[junit4:junit4] 2> 548286 T3032 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371987941956/'
[junit4:junit4] 2> 548324 T3032 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 548324 T3032 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 548324 T3032 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 548324 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 548325 T3032 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 548325 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 548325 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 548325 T3032 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 548326 T3032 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 548326 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 548329 T3032 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 548329 T3032 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33984/solr
[junit4:junit4] 2> 548329 T3032 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 548330 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 548331 T3072 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6d27e01f name:ZooKeeperConnection Watcher:127.0.0.1:33984 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 548331 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 548332 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 548335 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 548336 T3074 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3c69bbf7 name:ZooKeeperConnection Watcher:127.0.0.1:33984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 548336 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 548339 T3032 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 549341 T3032 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34080_
[junit4:junit4] 2> 549343 T3032 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34080_
[junit4:junit4] 2> 549344 T3061 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 549344 T3054 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 549344 T3074 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 549345 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 549348 T3075 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 549348 T3075 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 549680 T3055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 549681 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:38912_",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:38912__collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:38912"}
[junit4:junit4] 2> 549685 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34080_",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34080"}
[junit4:junit4] 2> 549686 T3055 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 549686 T3055 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 549690 T3074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 549691 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 549690 T3061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 550349 T3075 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371987941956/collection1
[junit4:junit4] 2> 550350 T3075 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 550350 T3075 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 550350 T3075 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 550352 T3075 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371987941956/collection1/'
[junit4:junit4] 2> 550352 T3075 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371987941956/collection1/lib/README' to classloader
[junit4:junit4] 2> 550352 T3075 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371987941956/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 550384 T3075 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 550427 T3075 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 550429 T3075 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 550432 T3075 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 550733 T3075 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 550734 T3075 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 550734 T3075 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 550739 T3075 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 550741 T3075 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 550757 T3075 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 550761 T3075 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 550765 T3075 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 550766 T3075 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 550766 T3075 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 550767 T3075 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 550768 T3075 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 550768 T3075 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 550768 T3075 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 550769 T3075 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371987941956/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/
[junit4:junit4] 2> 550769 T3075 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4afb9929
[junit4:junit4] 2> 550770 T3075 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 550770 T3075 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1
[junit4:junit4] 2> 550771 T3075 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index/
[junit4:junit4] 2> 550771 T3075 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 550772 T3075 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index
[junit4:junit4] 2> 550777 T3075 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 550777 T3075 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 550780 T3075 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 550781 T3075 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 550782 T3075 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 550783 T3075 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 550783 T3075 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 550784 T3075 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 550784 T3075 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 550785 T3075 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 550786 T3075 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 550788 T3075 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 550793 T3075 oass.SolrIndexSearcher.<init> Opening Searcher@32e7a8b3 main
[junit4:junit4] 2> 550793 T3075 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 550793 T3075 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 550798 T3076 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@32e7a8b3 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 550801 T3075 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 550802 T3075 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34080 collection:collection1 shard:shard1
[junit4:junit4] 2> 550803 T3075 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 550809 T3075 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 550812 T3075 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 550813 T3075 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 550813 T3075 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34080/collection1/
[junit4:junit4] 2> 550813 T3075 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 550814 T3075 oasc.SyncStrategy.syncToMe http://127.0.0.1:34080/collection1/ has no replicas
[junit4:junit4] 2> 550814 T3075 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:34080/collection1/
[junit4:junit4] 2> 550814 T3075 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 551198 T3055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 551210 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 551210 T3061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 551210 T3074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 551223 T3075 oasc.ZkController.register We are http://127.0.0.1:34080/collection1/ and leader is http://127.0.0.1:34080/collection1/
[junit4:junit4] 2> 551224 T3075 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:34080
[junit4:junit4] 2> 551224 T3075 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 551224 T3075 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 551225 T3075 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 551227 T3075 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 551228 T3032 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 551229 T3032 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 551230 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 551303 T3032 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 551305 T3032 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52561
[junit4:junit4] 2> 551306 T3032 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 551306 T3032 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 551306 T3032 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371987944970
[junit4:junit4] 2> 551307 T3032 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371987944970/solr.xml
[junit4:junit4] 2> 551307 T3032 oasc.CoreContainer.<init> New CoreContainer 1021245305
[junit4:junit4] 2> 551307 T3032 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371987944970/'
[junit4:junit4] 2> 551307 T3032 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371987944970/'
[junit4:junit4] 2> 551346 T3032 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 551346 T3032 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 551347 T3032 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 551347 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 551347 T3032 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 551347 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 551348 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 551348 T3032 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 551348 T3032 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 551348 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 551351 T3032 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 551351 T3032 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33984/solr
[junit4:junit4] 2> 551352 T3032 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 551352 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 551355 T3088 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e1a1876 name:ZooKeeperConnection Watcher:127.0.0.1:33984 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 551355 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 551357 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 551362 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 551363 T3090 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13dee31e name:ZooKeeperConnection Watcher:127.0.0.1:33984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 551363 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 551367 T3032 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 552370 T3032 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52561_
[junit4:junit4] 2> 552371 T3032 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52561_
[junit4:junit4] 2> 552373 T3074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 552373 T3054 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 552373 T3061 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 552373 T3090 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 552373 T3061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 552373 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 552374 T3074 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 552378 T3091 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 552378 T3091 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 552716 T3055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 552716 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:34080_",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:34080__collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:34080"}
[junit4:junit4] 2> 552718 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:52561_",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52561"}
[junit4:junit4] 2> 552719 T3055 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 552719 T3055 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 552721 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 552721 T3061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 552721 T3090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 552721 T3074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 553379 T3091 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371987944970/collection1
[junit4:junit4] 2> 553380 T3091 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 553381 T3091 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 553381 T3091 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 553383 T3091 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371987944970/collection1/'
[junit4:junit4] 2> 553384 T3091 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371987944970/collection1/lib/README' to classloader
[junit4:junit4] 2> 553385 T3091 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371987944970/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 553420 T3091 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 553450 T3091 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 553452 T3091 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 553456 T3091 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 553808 T3091 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 553808 T3091 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 553809 T3091 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 553815 T3091 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 553819 T3091 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 553830 T3091 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 553833 T3091 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 553836 T3091 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 553837 T3091 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 553837 T3091 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 553837 T3091 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 553838 T3091 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 553838 T3091 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 553839 T3091 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 553839 T3091 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371987944970/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/
[junit4:junit4] 2> 553839 T3091 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4afb9929
[junit4:junit4] 2> 553839 T3091 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 553840 T3091 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2
[junit4:junit4] 2> 553840 T3091 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index/
[junit4:junit4] 2> 553840 T3091 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 553840 T3091 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index
[junit4:junit4] 2> 553843 T3091 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@78d75261; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 553844 T3091 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 553845 T3091 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 553846 T3091 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 553846 T3091 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 553847 T3091 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 553847 T3091 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 553847 T3091 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 553847 T3091 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 553848 T3091 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 553848 T3091 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 553850 T3091 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 553853 T3091 oass.SolrIndexSearcher.<init> Opening Searcher@3a5c79be main
[junit4:junit4] 2> 553853 T3091 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 553853 T3091 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 553857 T3092 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3a5c79be main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 553858 T3091 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 553858 T3091 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52561 collection:collection1 shard:shard1
[junit4:junit4] 2> 553861 T3091 oasc.ZkController.register We are http://127.0.0.1:52561/collection1/ and leader is http://127.0.0.1:34080/collection1/
[junit4:junit4] 2> 553861 T3091 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52561
[junit4:junit4] 2> 553862 T3091 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 553862 T3091 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C620 name=collection1 org.apache.solr.core.SolrCore@50076f26 url=http://127.0.0.1:52561/collection1 node=127.0.0.1:52561_ C620_STATE=coll:collection1 core:collection1 props:{core=collection1, node_name=127.0.0.1:52561_, collection=collection1, state=down, base_url=http://127.0.0.1:52561}
[junit4:junit4] 2> 553862 T3093 C620 P52561 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 553863 T3093 C620 P52561 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 553863 T3091 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 553863 T3093 C620 P52561 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 553863 T3093 C620 P52561 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 553864 T3032 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 553864 T3032 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 553864 T3093 C620 P52561 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 553864 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 553870 T3067 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:52561__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 553933 T3032 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 553935 T3032 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58505
[junit4:junit4] 2> 553936 T3032 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 553936 T3032 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 553936 T3032 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371987947601
[junit4:junit4] 2> 553937 T3032 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371987947601/solr.xml
[junit4:junit4] 2> 553937 T3032 oasc.CoreContainer.<init> New CoreContainer 976008162
[junit4:junit4] 2> 553938 T3032 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371987947601/'
[junit4:junit4] 2> 553938 T3032 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371987947601/'
[junit4:junit4] 2> 553978 T3032 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 553978 T3032 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 553978 T3032 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 553978 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 553979 T3032 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 553979 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 553979 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 553979 T3032 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 553980 T3032 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 553980 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 553983 T3032 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 553983 T3032 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33984/solr
[junit4:junit4] 2> 553984 T3032 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 553984 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 553987 T3105 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1658e486 name:ZooKeeperConnection Watcher:127.0.0.1:33984 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 553987 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 553988 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 553992 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 553992 T3107 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68f19f94 name:ZooKeeperConnection Watcher:127.0.0.1:33984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 553993 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 553995 T3032 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 554225 T3055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 554226 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:52561_",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:52561__collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52561"}
[junit4:junit4] 2> 554230 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 554230 T3107 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 554230 T3090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 554230 T3074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 554230 T3061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 554871 T3067 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:52561__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 554871 T3067 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={core=collection1&version=2&wt=javabin&nodeName=127.0.0.1:52561_&coreNodeName=127.0.0.1:52561__collection1&state=recovering&onlyIfLeader=true&action=PREPRECOVERY&checkLive=true} status=0 QTime=1001
[junit4:junit4] 2> 554998 T3032 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58505_
[junit4:junit4] 2> 554999 T3032 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:58505_
[junit4:junit4] 2> 555001 T3090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 555001 T3074 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 555001 T3074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555001 T3054 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 555001 T3107 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 555001 T3061 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 555002 T3107 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555002 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555002 T3090 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 555002 T3061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555006 T3108 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 555006 T3108 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 555734 T3055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 555735 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:58505_",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58505"}
[junit4:junit4] 2> 555735 T3055 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 555735 T3055 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 555738 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555738 T3107 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555738 T3074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555738 T3090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 555738 T3061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 556007 T3108 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371987947601/collection1
[junit4:junit4] 2> 556008 T3108 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 556008 T3108 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 556008 T3108 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 556009 T3108 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371987947601/collection1/'
[junit4:junit4] 2> 556010 T3108 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371987947601/collection1/lib/README' to classloader
[junit4:junit4] 2> 556010 T3108 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371987947601/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 556041 T3108 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 556078 T3108 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 556079 T3108 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 556082 T3108 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 556411 T3108 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 556411 T3108 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 556412 T3108 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 556416 T3108 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 556418 T3108 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 556433 T3108 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 556436 T3108 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 556440 T3108 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 556441 T3108 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 556441 T3108 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 556441 T3108 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 556442 T3108 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 556442 T3108 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 556443 T3108 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 556443 T3108 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371987947601/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/
[junit4:junit4] 2> 556443 T3108 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4afb9929
[junit4:junit4] 2> 556444 T3108 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 556444 T3108 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3
[junit4:junit4] 2> 556444 T3108 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index/
[junit4:junit4] 2> 556445 T3108 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 556445 T3108 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index
[junit4:junit4] 2> 556450 T3108 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@804c63e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 556450 T3108 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 556452 T3108 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 556453 T3108 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 556453 T3108 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 556454 T3108 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 556455 T3108 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 556455 T3108 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 556456 T3108 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 556457 T3108 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 556457 T3108 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 556460 T3108 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 556465 T3108 oass.SolrIndexSearcher.<init> Opening Searcher@39b897c8 main
[junit4:junit4] 2> 556466 T3108 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 556467 T3108 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 556473 T3109 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@39b897c8 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 556476 T3108 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 556477 T3108 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:58505 collection:collection1 shard:shard1
[junit4:junit4] 2> 556480 T3108 oasc.ZkController.register We are http://127.0.0.1:58505/collection1/ and leader is http://127.0.0.1:34080/collection1/
[junit4:junit4] 2> 556480 T3108 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:58505
[junit4:junit4] 2> 556480 T3108 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 556480 T3108 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C621 name=collection1 org.apache.solr.core.SolrCore@2faa0431 url=http://127.0.0.1:58505/collection1 node=127.0.0.1:58505_ C621_STATE=coll:collection1 core:collection1 props:{core=collection1, node_name=127.0.0.1:58505_, collection=collection1, state=down, base_url=http://127.0.0.1:58505}
[junit4:junit4] 2> 556481 T3110 C621 P58505 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 556482 T3110 C621 P58505 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 556482 T3108 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 556482 T3110 C621 P58505 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 556482 T3110 C621 P58505 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 556483 T3032 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 556484 T3110 C621 P58505 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 556484 T3032 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 556484 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 556489 T3067 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:58505__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 556548 T3032 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 556550 T3032 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:36524
[junit4:junit4] 2> 556551 T3032 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 556551 T3032 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 556551 T3032 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371987950221
[junit4:junit4] 2> 556551 T3032 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371987950221/solr.xml
[junit4:junit4] 2> 556552 T3032 oasc.CoreContainer.<init> New CoreContainer 476007796
[junit4:junit4] 2> 556552 T3032 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371987950221/'
[junit4:junit4] 2> 556552 T3032 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371987950221/'
[junit4:junit4] 2> 556600 T3032 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 556601 T3032 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 556601 T3032 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 556601 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 556601 T3032 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 556602 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 556602 T3032 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 556602 T3032 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 556603 T3032 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 556603 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 556606 T3032 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 556607 T3032 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:33984/solr
[junit4:junit4] 2> 556607 T3032 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 556608 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 556610 T3122 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@33c58b7c name:ZooKeeperConnection Watcher:127.0.0.1:33984 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 556610 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 556611 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 556615 T3032 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 556616 T3124 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@139e010b name:ZooKeeperConnection Watcher:127.0.0.1:33984/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 556616 T3032 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 556619 T3032 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C622 name=collection1 org.apache.solr.core.SolrCore@50076f26 url=http://127.0.0.1:52561/collection1 node=127.0.0.1:52561_ C622_STATE=coll:collection1 core:collection1 props:{core=collection1, node_name=127.0.0.1:52561_, collection=collection1, shard=shard1, state=recovering, base_url=http://127.0.0.1:52561}
[junit4:junit4] 2> 556872 T3093 C622 P52561 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:34080/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 556873 T3093 C622 P52561 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:52561 START replicas=[http://127.0.0.1:34080/collection1/] nUpdates=100
[junit4:junit4] 2> 556873 T3093 C622 P52561 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 556873 T3093 C622 P52561 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 556873 T3093 C622 P52561 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 556873 T3093 C622 P52561 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 556873 T3093 C622 P52561 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 556873 T3093 C622 P52561 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:34080/collection1/. core=collection1
[junit4:junit4] 2> 556874 T3093 C622 P52561 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C623 name=collection1 org.apache.solr.core.SolrCore@23c16432 url=http://127.0.0.1:34080/collection1 node=127.0.0.1:34080_ C623_STATE=coll:collection1 core:collection1 props:{core=collection1, node_name=127.0.0.1:34080_, collection=collection1, shard=shard1, state=active, base_url=http://127.0.0.1:34080, leader=true}
[junit4:junit4] 2> 556878 T3069 C623 P34080 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&getVersions=100&wt=javabin&qt=/get&distrib=false} status=0 QTime=1
[junit4:junit4] 2> 556880 T3070 C623 P34080 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 556882 T3070 C623 P34080 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 556883 T3070 C623 P34080 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 556895 T3070 C623 P34080 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 556896 T3070 C623 P34080 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 556896 T3070 C623 P34080 oass.SolrIndexSearcher.<init> Opening Searcher@3a027e8c realtime
[junit4:junit4] 2> 556896 T3070 C623 P34080 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 556897 T3070 C623 P34080 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&softCommit=false&openSearcher=false&commit_end_point=true} {commit=} 0 17
[junit4:junit4] 2> 556898 T3093 C622 P52561 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 556898 T3093 C622 P52561 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 556922 T3070 C623 P34080 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 556923 T3070 C623 P34080 oasc.SolrCore.execute [collection1] webapp= path=/replication params={version=2&qt=/replication&wt=javabin&command=indexversion} status=0 QTime=24
[junit4:junit4] 2> 556923 T3093 C622 P52561 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 556923 T3093 C622 P52561 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 556924 T3093 C622 P52561 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 556926 T3069 C623 P34080 oasc.SolrCore.execute [collection1] webapp= path=/replication params={version=2&generation=2&qt=/replication&wt=javabin&command=filelist} status=0 QTime=1
[junit4:junit4] 2> 556926 T3093 C622 P52561 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 556927 T3093 C622 P52561 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index.20130623194550659
[junit4:junit4] 2> 556927 T3093 C622 P52561 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index.20130623194550659 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7aa241a4; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 556930 T3068 C623 P34080 oasc.SolrCore.execute [collection1] webapp= path=/replication params={generation=2&wt=filestream&file=segments_2&qt=/replication&command=filecontent&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 556932 T3093 C622 P52561 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 556936 T3093 C622 P52561 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 556937 T3093 C622 P52561 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 556940 T3093 C622 P52561 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@78d75261; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@78d75261; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 556941 T3093 C622 P52561 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 556941 T3093 C622 P52561 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 556942 T3093 C622 P52561 oass.SolrIndexSearcher.<init> Opening Searcher@4ebceb03 main
[junit4:junit4] 2> 556944 T3092 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4ebceb03 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 556944 T3093 C622 P52561 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index.20130623194550659 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index.20130623194550659;done=true>>]
[junit4:junit4] 2> 556945 T3093 C622 P52561 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index.20130623194550659
[junit4:junit4] 2> 556945 T3093 C622 P52561 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index.20130623194550659
[junit4:junit4] 2> 556946 T3093 C622 P52561 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 556946 T3093 C622 P52561 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 556946 T3093 C622 P52561 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 556946 T3093 C622 P52561 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 556949 T3093 C622 P52561 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 557243 T3055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 557244 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:58505_",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:58505__collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58505"}
[junit4:junit4] 2> 557247 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:52561_",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:52561__collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52561"}
[junit4:junit4] 2> 557266 T3124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 557266 T3107 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 557266 T3090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 557266 T3074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 557266 T3061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 557266 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 557489 T3067 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:58505__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 557489 T3067 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={core=collection1&version=2&wt=javabin&nodeName=127.0.0.1:58505_&coreNodeName=127.0.0.1:58505__collection1&state=recovering&onlyIfLeader=true&action=PREPRECOVERY&checkLive=true} status=0 QTime=1000
[junit4:junit4] 2> 557621 T3032 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36524_
[junit4:junit4] 2> 557622 T3032 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36524_
[junit4:junit4] 2> 557623 T3124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 557623 T3090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 557624 T3054 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 557623 T3107 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 557624 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 557624 T3074 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 557624 T3061 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 557625 T3061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 557625 T3074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 557627 T3090 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 557627 T3124 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 557627 T3107 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 557630 T3127 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 557630 T3127 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 558772 T3055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 558774 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:36524_",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:36524"}
[junit4:junit4] 2> 558774 T3055 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 558774 T3055 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 558786 T3124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 558786 T3090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 558786 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 558786 T3107 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 558786 T3074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 558786 T3061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> ASYNC NEW_CORE C624 name=collection1 org.apache.solr.core.SolrCore@2faa0431 url=http://127.0.0.1:58505/collection1 node=127.0.0.1:58505_ C624_STATE=coll:collection1 core:collection1 props:{core=collection1, node_name=127.0.0.1:58505_, collection=collection1, shard=shard1, state=recovering, base_url=http://127.0.0.1:58505}
[junit4:junit4] 2> 559490 T3110 C624 P58505 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:34080/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 559491 T3110 C624 P58505 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:58505 START replicas=[http://127.0.0.1:34080/collection1/] nUpdates=100
[junit4:junit4] 2> 559491 T3110 C624 P58505 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 559491 T3110 C624 P58505 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 559492 T3110 C624 P58505 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 559492 T3110 C624 P58505 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 559492 T3110 C624 P58505 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 559492 T3110 C624 P58505 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:34080/collection1/. core=collection1
[junit4:junit4] 2> 559492 T3110 C624 P58505 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 559493 T3069 C623 P34080 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&getVersions=100&wt=javabin&qt=/get&distrib=false} status=0 QTime=0
[junit4:junit4] 2> 559498 T3068 C623 P34080 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 559505 T3068 C623 P34080 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 559506 T3068 C623 P34080 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 559507 T3068 C623 P34080 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 559507 T3068 C623 P34080 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&softCommit=false&openSearcher=false&commit_end_point=true} {commit=} 0 9
[junit4:junit4] 2> 559508 T3110 C624 P58505 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 559508 T3110 C624 P58505 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 559510 T3067 C623 P34080 oasc.SolrCore.execute [collection1] webapp= path=/replication params={version=2&qt=/replication&wt=javabin&command=indexversion} status=0 QTime=0
[junit4:junit4] 2> 559511 T3110 C624 P58505 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 559511 T3110 C624 P58505 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 559511 T3110 C624 P58505 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 559513 T3069 C623 P34080 oasc.SolrCore.execute [collection1] webapp= path=/replication params={version=2&generation=3&qt=/replication&wt=javabin&command=filelist} status=0 QTime=1
[junit4:junit4] 2> 559513 T3110 C624 P58505 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 559514 T3110 C624 P58505 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index.20130623194553246
[junit4:junit4] 2> 559515 T3110 C624 P58505 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index.20130623194553246 lockFactory=org.apache.lucene.store.NativeFSLockFactory@89873cf; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 559517 T3068 C623 P34080 oasc.SolrCore.execute [collection1] webapp= path=/replication params={generation=3&wt=filestream&file=segments_3&qt=/replication&command=filecontent&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 559518 T3110 C624 P58505 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 559521 T3110 C624 P58505 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 559521 T3110 C624 P58505 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 559523 T3110 C624 P58505 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@804c63e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@804c63e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 559523 T3110 C624 P58505 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 559524 T3110 C624 P58505 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 559524 T3110 C624 P58505 oass.SolrIndexSearcher.<init> Opening Searcher@b2512e main
[junit4:junit4] 2> 559525 T3109 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@b2512e main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 559526 T3110 C624 P58505 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index.20130623194553246 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index.20130623194553246;done=true>>]
[junit4:junit4] 2> 559526 T3110 C624 P58505 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index.20130623194553246
[junit4:junit4] 2> 559526 T3110 C624 P58505 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index.20130623194553246
[junit4:junit4] 2> 559526 T3110 C624 P58505 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 559526 T3110 C624 P58505 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 559527 T3110 C624 P58505 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 559527 T3110 C624 P58505 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 559528 T3110 C624 P58505 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 559631 T3127 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371987950221/collection1
[junit4:junit4] 2> 559632 T3127 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 559632 T3127 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 559633 T3127 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 559634 T3127 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371987950221/collection1/'
[junit4:junit4] 2> 559635 T3127 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371987950221/collection1/lib/README' to classloader
[junit4:junit4] 2> 559635 T3127 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371987950221/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 559676 T3127 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 559705 T3127 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 559707 T3127 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 559710 T3127 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 560174 T3127 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 560174 T3127 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 560175 T3127 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 560181 T3127 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 560184 T3127 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 560196 T3127 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 560199 T3127 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 560202 T3127 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 560203 T3127 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 560203 T3127 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 560203 T3127 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 560204 T3127 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 560204 T3127 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 560204 T3127 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 560204 T3127 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371987950221/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/
[junit4:junit4] 2> 560204 T3127 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4afb9929
[junit4:junit4] 2> 560205 T3127 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 560205 T3127 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4
[junit4:junit4] 2> 560205 T3127 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index/
[junit4:junit4] 2> 560206 T3127 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 560206 T3127 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index
[junit4:junit4] 2> 560227 T3127 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a7a5b5f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 560227 T3127 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 560229 T3127 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 560229 T3127 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 560230 T3127 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 560230 T3127 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 560231 T3127 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 560231 T3127 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 560232 T3127 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 560233 T3127 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 560233 T3127 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 560234 T3127 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 560237 T3127 oass.SolrIndexSearcher.<init> Opening Searcher@67f1c431 main
[junit4:junit4] 2> 560238 T3127 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 560238 T3127 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 560243 T3130 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67f1c431 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 560245 T3127 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 560245 T3127 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:36524 collection:collection1 shard:shard1
[junit4:junit4] 2> 560248 T3127 oasc.ZkController.register We are http://127.0.0.1:36524/collection1/ and leader is http://127.0.0.1:34080/collection1/
[junit4:junit4] 2> 560248 T3127 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36524
[junit4:junit4] 2> 560248 T3127 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 560249 T3127 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C625 name=collection1 org.apache.solr.core.SolrCore@1439b349 url=http://127.0.0.1:36524/collection1 node=127.0.0.1:36524_ C625_STATE=coll:collection1 core:collection1 props:{core=collection1, node_name=127.0.0.1:36524_, collection=collection1, state=down, base_url=http://127.0.0.1:36524}
[junit4:junit4] 2> 560249 T3131 C625 P36524 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 560249 T3127 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 560250 T3131 C625 P36524 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 560251 T3131 C625 P36524 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 560251 T3131 C625 P36524 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 560251 T3032 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 560252 T3131 C625 P36524 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 560252 T3032 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 560252 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 560255 T3069 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:36524__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 560256 T3032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 560258 T3032 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 560259 T3032 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 560259 T3032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 560292 T3055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 560292 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:58505_",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:58505__collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:58505"}
[junit4:junit4] 2> 560295 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:36524_",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:36524__collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:36524"}
[junit4:junit4] 2> 560298 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 560298 T3090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 560298 T3074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 560298 T3107 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 560298 T3124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 560298 T3061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 561256 T3069 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:36524__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 561256 T3069 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={core=collection1&version=2&wt=javabin&nodeName=127.0.0.1:36524_&coreNodeName=127.0.0.1:36524__collection1&state=recovering&onlyIfLeader=true&action=PREPRECOVERY&checkLive=true} status=0 QTime=1001
[junit4:junit4] 2> 561261 T3032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 562262 T3032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C625_STATE=coll:collection1 core:collection1 props:{core=collection1, node_name=127.0.0.1:36524_, collection=collection1, shard=shard1, state=recovering, base_url=http://127.0.0.1:36524}
[junit4:junit4] 2> 563257 T3131 C625 P36524 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:34080/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 563257 T3131 C625 P36524 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:36524 START replicas=[http://127.0.0.1:34080/collection1/] nUpdates=100
[junit4:junit4] 2> 563258 T3131 C625 P36524 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 563258 T3131 C625 P36524 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 563258 T3131 C625 P36524 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 563258 T3131 C625 P36524 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 563258 T3131 C625 P36524 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 563259 T3131 C625 P36524 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:34080/collection1/. core=collection1
[junit4:junit4] 2> 563259 T3131 C625 P36524 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C626 name=collection1 org.apache.solr.core.SolrCore@23c16432 url=http://127.0.0.1:34080/collection1 node=127.0.0.1:34080_ C626_STATE=coll:collection1 core:collection1 props:{core=collection1, node_name=127.0.0.1:34080_, collection=collection1, shard=shard1, state=active, base_url=http://127.0.0.1:34080, leader=true}
[junit4:junit4] 2> 563260 T3069 C626 P34080 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&getVersions=100&wt=javabin&qt=/get&distrib=false} status=0 QTime=0
[junit4:junit4] 2> 563264 T3070 C626 P34080 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 563264 T3032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 563275 T3070 C626 P34080 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 563276 T3070 C626 P34080 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 563277 T3070 C626 P34080 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 563277 T3070 C626 P34080 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&softCommit=false&openSearcher=false&commit_end_point=true} {commit=} 0 13
[junit4:junit4] 2> 563279 T3131 C625 P36524 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 563279 T3131 C625 P36524 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 563281 T3069 C626 P34080 oasc.SolrCore.execute [collection1] webapp= path=/replication params={version=2&qt=/replication&wt=javabin&command=indexversion} status=0 QTime=0
[junit4:junit4] 2> 563282 T3131 C625 P36524 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 563282 T3131 C625 P36524 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 563282 T3131 C625 P36524 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 563285 T3068 C626 P34080 oasc.SolrCore.execute [collection1] webapp= path=/replication params={version=2&generation=4&qt=/replication&wt=javabin&command=filelist} status=0 QTime=1
[junit4:junit4] 2> 563285 T3131 C625 P36524 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 563286 T3131 C625 P36524 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index.20130623194557019
[junit4:junit4] 2> 563287 T3131 C625 P36524 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index.20130623194557019 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2689b772; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 563290 T3070 C626 P34080 oasc.SolrCore.execute [collection1] webapp= path=/replication params={generation=4&wt=filestream&file=segments_4&qt=/replication&command=filecontent&checksum=true} status=0 QTime=0
[junit4:junit4] 2> 563292 T3131 C625 P36524 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 563294 T3131 C625 P36524 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 563294 T3131 C625 P36524 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 563297 T3131 C625 P36524 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a7a5b5f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a7a5b5f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 563297 T3131 C625 P36524 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 563298 T3131 C625 P36524 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 563298 T3131 C625 P36524 oass.SolrIndexSearcher.<init> Opening Searcher@517faef1 main
[junit4:junit4] 2> 563299 T3130 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@517faef1 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 563300 T3131 C625 P36524 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index.20130623194557019 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index.20130623194557019;done=true>>]
[junit4:junit4] 2> 563300 T3131 C625 P36524 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index.20130623194557019
[junit4:junit4] 2> 563300 T3131 C625 P36524 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index.20130623194557019
[junit4:junit4] 2> 563301 T3131 C625 P36524 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 563301 T3131 C625 P36524 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 563301 T3131 C625 P36524 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 563301 T3131 C625 P36524 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 563303 T3131 C625 P36524 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 563309 T3055 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 563310 T3055 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "node_name":"127.0.0.1:36524_",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:36524__collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:36524"}
[junit4:junit4] 2> 563313 T3074 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 563313 T3054 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 563313 T3061 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 563313 T3124 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 563313 T3090 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 563313 T3107 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 564266 T3032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 564267 T3032 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C627 name=collection1 org.apache.solr.core.SolrCore@24f8e2e5 url=http://127.0.0.1:38912/collection1 node=127.0.0.1:38912_ C627_STATE=coll:control_collection core:collection1 props:{core=collection1, node_name=127.0.0.1:38912_, collection=control_collection, shard=shard1, state=active, base_url=http://127.0.0.1:38912, leader=true}
[junit4:junit4] 2> 564277 T3048 C627 P38912 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 564280 T3048 C627 P38912 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2adb1318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 564281 T3048 C627 P38912 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 564296 T3048 C627 P38912 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2adb1318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2adb1318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 564296 T3048 C627 P38912 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 564297 T3048 C627 P38912 oass.SolrIndexSearcher.<init> Opening Searcher@35c81fde main
[junit4:junit4] 2> 564298 T3048 C627 P38912 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 564299 T3058 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@35c81fde main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 564300 T3048 C627 P38912 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> ASYNC NEW_CORE C628 name=collection1 org.apache.solr.core.SolrCore@2faa0431 url=http://127.0.0.1:58505/collection1 node=127.0.0.1:58505_ C628_STATE=coll:collection1 core:collection1 props:{core=collection1, node_name=127.0.0.1:58505_, collection=collection1, shard=shard1, state=active, base_url=http://127.0.0.1:58505}
[junit4:junit4] 2> 564315 T3103 C628 P58505 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:34080/collection1/, StdNode: http://127.0.0.1:52561/collection1/, StdNode: http://127.0.0.1:58505/collection1/, StdNode: http://127.0.0.1:36524/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 564321 T3070 C626 P34080 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 564321 T3101 C628 P58505 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C625_STATE=coll:collection1 core:collection1 props:{core=collection1, node_name=127.0.0.1:36524_, collection=collection1, shard=shard1, state=active, base_url=http://127.0.0.1:36524}
[junit4:junit4] 2> 564330 T3120 C625 P36524 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C629 name=collection1 org.apache.solr.core.SolrCore@50076f26 url=http://127.0.0.1:52561/collection1 node=127.0.0.1:52561_ C629_STATE=coll:collection1 core:collection1 props:{core=collection1, node_name=127.0.0.1:52561_, collection=collection1, shard=shard1, state=active, base_url=http://127.0.0.1:52561}
[junit4:junit4] 2> 564330 T3085 C629 P52561 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 564335 T3070 C626 P34080 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 564336 T3070 C626 P34080 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 564337 T3070 C626 P34080 oass.SolrIndexSearcher.<init> Opening Searcher@30cd3947 main
[junit4:junit4] 2> 564337 T3070 C626 P34080 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 564339 T3076 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@30cd3947 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 564339 T3070 C626 P34080 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&expungeDeletes=false&softCommit=false&commit_end_point=true} {commit=} 0 19
[junit4:junit4] 2> 564346 T3101 C628 P58505 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@804c63e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@804c63e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 564347 T3101 C628 P58505 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 564347 T3101 C628 P58505 oass.SolrIndexSearcher.<init> Opening Searcher@4130c98f main
[junit4:junit4] 2> 564348 T3101 C628 P58505 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 564349 T3109 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4130c98f main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 564349 T3101 C628 P58505 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&expungeDeletes=false&softCommit=false&commit_end_point=true} {commit=} 0 28
[junit4:junit4] 2> 564366 T3085 C629 P52561 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@78d75261; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@78d75261; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 564366 T3120 C625 P36524 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a7a5b5f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a7a5b5f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 564367 T3120 C625 P36524 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 564366 T3085 C629 P52561 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 564368 T3120 C625 P36524 oass.SolrIndexSearcher.<init> Opening Searcher@33e80761 main
[junit4:junit4] 2> 564370 T3085 C629 P52561 oass.SolrIndexSearcher.<init> Opening Searcher@27f20aa6 main
[junit4:junit4] 2> 564370 T3120 C625 P36524 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 564371 T3085 C629 P52561 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 564372 T3092 oasc.JmxMonitoredMap.put WARN Failed to register info bean: fieldValueCache javax.management.InstanceAlreadyExistsException: solr/collection1:id=org.apache.solr.search.FastLRUCache,type=fieldValueCache
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:302)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1846)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1723)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2>
[junit4:junit4] 2> 564372 T3130 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@33e80761 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 564373 T3092 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@27f20aa6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 564373 T3120 C625 P36524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&expungeDeletes=false&softCommit=false&commit_end_point=true} {commit=} 0 43
[junit4:junit4] 2> 564374 T3085 C629 P52561 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&expungeDeletes=false&softCommit=false&commit_end_point=true} {commit=} 0 44
[junit4:junit4] 2> 564375 T3103 C628 P58505 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&softCommit=false} {commit=} 0 60
[junit4:junit4] 2> 564377 T3032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 564380 T3069 C626 P34080 oasc.SolrCore.execute [collection1] webapp= path=/select params={rows=0&version=2&q=*:*&tests=checkShardConsistency&wt=javabin&distrib=false} hits=0 status=0 QTime=0
[junit4:junit4] 2> 564383 T3084 C629 P52561 oasc.SolrCore.execute [collection1] webapp= path=/select params={rows=0&version=2&q=*:*&tests=checkShardConsistency&wt=javabin&distrib=false} hits=0 status=0 QTime=0
[junit4:junit4] 2> 564386 T3100 C628 P58505 oasc.SolrCore.execute [collection1] webapp= path=/select params={rows=0&version=2&q=*:*&tests=checkShardConsistency&wt=javabin&distrib=false} hits=0 status=0 QTime=0
[junit4:junit4] 2> 564389 T3117 C625 P36524 oasc.SolrCore.execute [collection1] webapp= path=/select params={rows=0&version=2&q=*:*&tests=checkShardConsistency&wt=javabin&distrib=false} hits=0 status=0 QTime=1
[junit4:junit4] 2> 566393 T3048 C627 P38912 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {deleteByQuery=*:* (-1438633647276032000)} 0 1
[junit4:junit4] 2> 566403 T3101 C628 P58505 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&update.distrib=FROMLEADER&wt=javabin&_version_=-1438633647282323456&update.from=http://127.0.0.1:34080/collection1/} {deleteByQuery=*:* (-1438633647282323456)} 0 2
[junit4:junit4] 2> 566403 T3118 C625 P36524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&update.distrib=FROMLEADER&wt=javabin&_version_=-1438633647282323456&update.from=http://127.0.0.1:34080/collection1/} {deleteByQuery=*:* (-1438633647282323456)} 0 2
[junit4:junit4] 2> 566405 T3086 C629 P52561 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&update.distrib=FROMLEADER&wt=javabin&_version_=-1438633647282323456&update.from=http://127.0.0.1:34080/collection1/} {deleteByQuery=*:* (-1438633647282323456)} 0 1
[junit4:junit4] 2> 566406 T3069 C626 P34080 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&update.distrib=TOLEADER&wt=javabin} {deleteByQuery=*:* (-1438633647282323456)} 0 9
[junit4:junit4] 2> 566406 T3084 C629 P52561 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {deleteByQuery=*:*} 0 11
[junit4:junit4] 2> 566415 T3047 C627 P38912 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[0 (1438633647299100672)]} 0 2
[junit4:junit4] 2> 566425 T3119 C625 P36524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&distrib.from=http://127.0.0.1:34080/collection1/&update.distrib=FROMLEADER&wt=javabin} {add=[0 (1438633647305392128)]} 0 1
[junit4:junit4] 2> 566425 T3103 C628 P58505 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&distrib.from=http://127.0.0.1:34080/collection1/&update.distrib=FROMLEADER&wt=javabin} {add=[0 (1438633647305392128)]} 0 1
[junit4:junit4] 2> 566430 T3085 C629 P52561 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&distrib.from=http://127.0.0.1:34080/collection1/&update.distrib=FROMLEADER&wt=javabin} {add=[0 (1438633647305392128)]} 0 1
[junit4:junit4] 2> 566431 T3068 C626 P34080 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&distrib.from=http://127.0.0.1:52561/collection1/&update.distrib=TOLEADER&wt=javabin} {add=[0 (1438633647305392128)]} 0 11
[junit4:junit4] 2> 566432 T3083 C629 P52561 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[0]} 0 14
[junit4:junit4] 2> 566435 T3050 C627 P38912 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1 (1438633647320072192)]} 0 1
[junit4:junit4] 2> 566447 T3117 C625 P36524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&distrib.from=http://127.0.0.1:34080/collection1/&update.distrib=FROMLEADER&wt=javabin} {add=[1 (1438633647330557952)]} 0 1
[junit4:junit4] 2> 566447 T3102 C628 P58505 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&distrib.from=http://127.0.0.1:34080/collection1/&update.distrib=FROMLEADER&wt=javabin} {add=[1 (1438633647330557952)]} 0 1
[junit4:junit4] 2> 566449 T3086 C629 P52561 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&distrib.from=http://127.0.0.1:34080/collection1/&update.distrib=FROMLEADER&wt=javabin} {add=[1 (1438633647330557952)]} 0 1
[junit4:junit4] 2> 566449 T3067 C626 P34080 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&distrib.from=http://127.0.0.1:36524/collection1/&update.distrib=TOLEADER&wt=javabin} {add=[1 (1438633647330557952)]} 0 5
[junit4:junit4] 2> 566450 T3120 C625 P36524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[1]} 0 14
[junit4:junit4] 2> 566453 T3049 C627 P38912 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[2 (1438633647338946560)]} 0 1
[junit4:junit4] 2> 566458 T3069 C626 P34080 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:52561/collection1/]
[junit4:junit4] 2> 566458 T3069 C626 P34080 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:52561/collection1/ against:[http://127.0.0.1:52561/collection1/] result:true
[junit4:junit4] 2> 566459 T3069 C626 P34080 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:58505/collection1/ against:[http://127.0.0.1:52561/collection1/] result:false
[junit4:junit4] 2> 566459 T3069 C626 P34080 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:36524/collection1/ against:[http://127.0.0.1:52561/collection1/] result:false
[junit4:junit4] 2> 566465 T3100 C628 P58505 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&test.distrib.skip.servers=http://127.0.0.1:52561/collection1/&distrib.from=http://127.0.0.1:34080/collection1/&update.distrib=FROMLEADER&wt=javabin} {add=[2 (1438633647346286592)]} 0 1
[junit4:junit4] 2> 566467 T3119 C625 P36524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&test.distrib.skip.servers=http://127.0.0.1:52561/collection1/&distrib.from=http://127.0.0.1:34080/collection1/&update.distrib=FROMLEADER&wt=javabin} {add=[2 (1438633647346286592)]} 0 1
[junit4:junit4] 2> 566468 T3069 C626 P34080 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&test.distrib.skip.servers=http://127.0.0.1:52561/collection1/&distrib.from=http://127.0.0.1:36524/collection1/&update.distrib=TOLEADER&wt=javabin} {add=[2 (1438633647346286592)]} 0 10
[junit4:junit4] 2> 566468 T3118 C625 P36524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&test.distrib.skip.servers=http://127.0.0.1:52561/collection1/&wt=javabin} {add=[2]} 0 13
[junit4:junit4] 2> 566472 T3048 C627 P38912 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin} {add=[3 (1438633647358869504)]} 0 1
[junit4:junit4] 2> 566476 T3068 C626 P34080 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:52561/collection1/]
[junit4:junit4] 2> 566477 T3068 C626 P34080 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:52561/collection1/ against:[http://127.0.0.1:52561/collection1/] result:true
[junit4:junit4] 2> 566477 T3068 C626 P34080 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:58505/collection1/ against:[http://127.0.0.1:52561/collection1/] result:false
[junit4:junit4] 2> 566478 T3068 C626 P34080 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:36524/collection1/ against:[http://127.0.0.1:52561/collection1/] result:false
[junit4:junit4] 2> 566481 T3117 C625 P36524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&test.distrib.skip.servers=http://127.0.0.1:52561/collection1/&distrib.from=http://127.0.0.1:34080/collection1/&update.distrib=FROMLEADER&wt=javabin} {add=[3 (1438633647366209536)]} 0 0
[junit4:junit4] 2> 566483 T3101 C628 P58505 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&test.distrib.skip.servers=http://127.0.0.1:52561/collection1/&distrib.from=http://127.0.0.1:34080/collection1/&update.distrib=FROMLEADER&wt=javabin} {add=[3 (1438633647366209536)]} 0 1
[junit4:junit4] 2> 566483 T3068 C626 P34080 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&test.distrib.skip.servers=http://127.0.0.1:52561/collection1/&distrib.from=http://127.0.0.1:52561/collection1/&update.distrib=TOLEADER&wt=javabin} {add=[3 (1438633647366209536)]} 0 7
[junit4:junit4] 2> 566484 T3084 C629 P52561 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&test.distrib.skip.servers=http://127.0.0.1:52561/collection1/&test.distrib.skip.servers=http://127.0.0.1:58505/collection1/&wt=javabin} {add=[3]} 0 10
[junit4:junit4] 2> 566486 T3047 C627 P38912 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 566506 T3047 C627 P38912 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2adb1318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2adb1318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 566506 T3047 C627 P38912 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 566509 T3047 C627 P38912 oass.SolrIndexSearcher.<init> Opening Searcher@540f8f26 main
[junit4:junit4] 2> 566509 T3047 C627 P38912 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 566510 T3058 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@540f8f26 main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 566510 T3047 C627 P38912 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 566513 T3085 C629 P52561 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:34080/collection1/, StdNode: http://127.0.0.1:52561/collection1/, StdNode: http://127.0.0.1:58505/collection1/, StdNode: http://127.0.0.1:36524/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 566515 T3070 C626 P34080 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 566522 T3086 C629 P52561 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 566523 T3103 C628 P58505 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 566525 T3120 C625 P36524 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 566566 T3086 C629 P52561 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@78d75261; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@78d75261; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 566566 T3070 C626 P34080 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 566567 T3070 C626 P34080 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 566566 T3120 C625 P36524 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a7a5b5f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a7a5b5f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 566568 T3120 C625 P36524 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 566566 T3103 C628 P58505 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@804c63e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@804c63e; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 566566 T3086 C629 P52561 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 566568 T3103 C628 P58505 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 566577 T3120 C625 P36524 oass.SolrIndexSearcher.<init> Opening Searcher@67bdf0d6 main
[junit4:junit4] 2> 566578 T3120 C625 P36524 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 566578 T3070 C626 P34080 oass.SolrIndexSearcher.<init> Opening Searcher@65d2358a main
[junit4:junit4] 2> 566578 T3086 C629 P52561 oass.SolrIndexSearcher.<init> Opening Searcher@6aee9fff main
[junit4:junit4] 2> 566578 T3070 C626 P34080 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 566578 T3130 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@67bdf0d6 main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 566579 T3076 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@65d2358a main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 566580 T3092 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6aee9fff main{StandardDirectoryReader(segments_4:3:nrt _0(4.4):c2)}
[junit4:junit4] 2> 566579 T3086 C629 P52561 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 566580 T3086 C629 P52561 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&expungeDeletes=false&softCommit=false&commit_end_point=true} {commit=} 0 58
[junit4:junit4] 2> 566580 T3070 C626 P34080 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&expungeDeletes=false&softCommit=false&commit_end_point=true} {commit=} 0 65
[junit4:junit4] 2> 566580 T3120 C625 P36524 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&expungeDeletes=false&softCommit=false&commit_end_point=true} {commit=} 0 55
[junit4:junit4] 2> 566580 T3103 C628 P58505 oass.SolrIndexSearcher.<init> Opening Searcher@672155cc main
[junit4:junit4] 2> 566582 T3109 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@672155cc main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 566583 T3103 C628 P58505 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 566583 T3103 C628 P58505 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&expungeDeletes=false&softCommit=false&commit_end_point=true} {commit=} 0 60
[junit4:junit4] 2> 566584 T3085 C629 P52561 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&softCommit=false} {commit=} 0 71
[junit4:junit4] 2> 566585 T3032 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 566585 T3032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 566586 T3032 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 566589 T3067 C626 P34080 oasc.SolrCore.execute [collection1] webapp= path=/select params={rows=0&version=2&q=*:*&tests=checkShardConsistency&wt=javabin&distrib=false} hits=4 status=0 QTime=1
[junit4:junit4] 2> 566591 T3084 C629 P52561 oasc.SolrCore.execute [collection1] webapp= path=/select params={rows=0&version=2&q=*:*&tests=checkShardConsistency&wt=javabin&distrib=false} hits=2 status=0 QTime=0
[junit4:junit4] 2> 566594 T3102 C628 P58505 oasc.SolrCore.execute [collection1] webapp= path=/select params={rows=0&version=2&q=*:*&tests=checkShardConsistency&wt=javabin&distrib=false} hits=4 status=0 QTime=0
[junit4:junit4] 2> 566596 T3119 C625 P36524 oasc.SolrCore.execute [collection1] webapp= path=/select params={rows=0&version=2&q=*:*&tests=checkShardConsistency&wt=javabin&distrib=false} hits=4 status=0 QTime=0
[junit4:junit4] 2> 566597 T3032 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 566602 T3100 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : version=2&wt=javabin&collection=collection1&shard=shard1&action=SYNCSHARD
[junit4:junit4] 2> 566603 T3100 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 566608 T3068 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 566608 T3068 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:34080/collection1/
[junit4:junit4] 2> 566609 T3068 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34080 START replicas=[http://127.0.0.1:52561/collection1/, http://127.0.0.1:58505/collection1/, http://127.0.0.1:36524/collection1/] nUpdates=100
[junit4:junit4] 2> 566611 T3086 C629 P52561 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&getVersions=100&wt=javabin&qt=/get&distrib=false} status=0 QTime=0
[junit4:junit4] 2> 566611 T3117 C625 P36524 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&getVersions=100&wt=javabin&qt=/get&distrib=false} status=0 QTime=0
[junit4:junit4] 2> 566611 T3103 C628 P58505 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&getVersions=100&wt=javabin&qt=/get&distrib=false} status=0 QTime=0
[junit4:junit4] 2> 566612 T3068 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:34080 Received 3 versions from 127.0.0.1:52561/collection1/
[junit4:junit4] 2> 566612 T3068 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:34080 Our versions are newer. ourLowThreshold=1438633647282323456 otherHigh=1438633647330557952
[junit4:junit4] 2> 566613 T3068 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:34080 Received 5 versions from 127.0.0.1:36524/collection1/
[junit4:junit4] 2> 566613 T3068 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:34080 Our versions are newer. ourLowThreshold=1438633647282323456 otherHigh=1438633647346286592
[junit4:junit4] 2> 566614 T3068 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:34080 Received 5 versions from 127.0.0.1:58505/collection1/
[junit4:junit4] 2> 566614 T3068 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:34080 Our versions are newer. ourLowThreshold=1438633647282323456 otherHigh=1438633647346286592
[junit4:junit4] 2> 566614 T3068 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:34080 DONE. sync succeeded
[junit4:junit4] 2> 566614 T3068 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 566615 T3068 oasc.SyncStrategy.syncToMe http://127.0.0.1:34080/collection1/: try and ask http://127.0.0.1:52561/collection1/ to sync
[junit4:junit4] 2> 566615 T3068 oasc.SyncStrategy.syncToMe http://127.0.0.1:34080/collection1/: try and ask http://127.0.0.1:58505/collection1/ to sync
[junit4:junit4] 2> 566616 T3068 oasc.SyncStrategy.syncToMe http://127.0.0.1:34080/collection1/: try and ask http://127.0.0.1:36524/collection1/ to sync
[junit4:junit4] 2> 566617 T3084 C629 P52561 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:52561 START replicas=[http://127.0.0.1:34080/collection1/] nUpdates=100
[junit4:junit4] 2> 566618 T3119 C625 P36524 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:36524 START replicas=[http://127.0.0.1:34080/collection1/] nUpdates=100
[junit4:junit4] 2> 566619 T3101 C628 P58505 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:58505 START replicas=[http://127.0.0.1:34080/collection1/] nUpdates=100
[junit4:junit4] 2> 566620 T3070 C626 P34080 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&getVersions=100&wt=javabin&qt=/get&distrib=false} status=0 QTime=0
[junit4:junit4] 2> 566621 T3084 C629 P52561 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:52561 Received 5 versions from 127.0.0.1:34080/collection1/
[junit4:junit4] 2> 566621 T3070 C626 P34080 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&getVersions=100&wt=javabin&qt=/get&distrib=false} status=0 QTime=0
[junit4:junit4] 2> 566621 T3084 C629 P52561 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:52561 Requesting updates from 127.0.0.1:34080/collection1/n=2 versions=[1438633647366209536, 1438633647346286592]
[junit4:junit4] 2> 566621 T3119 C625 P36524 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:36524 Received 5 versions from 127.0.0.1:34080/collection1/
[junit4:junit4] 2> 566622 T3119 C625 P36524 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:36524 Our versions are newer. ourLowThreshold=1438633647282323456 otherHigh=1438633647346286592
[junit4:junit4] 2> 566622 T3119 C625 P36524 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:36524 DONE. sync succeeded
[junit4:junit4] 2> 566622 T3119 C625 P36524 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&getVersions=100&wt=javabin&qt=/get&distrib=false&sync=http://127.0.0.1:34080/collection1/} status=0 QTime=4
[junit4:junit4] 2> 566623 T3069 C626 P34080 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&getVersions=100&wt=javabin&qt=/get&distrib=false} status=0 QTime=1
[junit4:junit4] 2> 566623 T3068 oasc.SyncStrategy.syncToMe http://127.0.0.1:34080/collection1/: sync completed with http://127.0.0.1:36524/collection1/
[junit4:junit4] 2> 566624 T3067 C626 P34080 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&wt=javabin&qt=/get&getUpdates=1438633647366209536,1438633647346286592&distrib=false} status=0 QTime=0
[junit4:junit4] 2> 566625 T3101 C628 P58505 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:58505 Received 5 versions from 127.0.0.1:34080/collection1/
[junit4:junit4] 2> 566625 T3101 C628 P58505 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:58505 Our versions are newer. ourLowThreshold=1438633647282323456 otherHigh=1438633647346286592
[junit4:junit4] 2> 566626 T3101 C628 P58505 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:58505 DONE. sync succeeded
[junit4:junit4] 2> 566626 T3101 C628 P58505 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&getVersions=100&wt=javabin&qt=/get&distrib=false&sync=http://127.0.0.1:34080/collection1/} status=0 QTime=7
[junit4:junit4] 2> 566626 T3068 oasc.SyncStrategy.syncToMe http://127.0.0.1:34080/collection1/: sync completed with http://127.0.0.1:58505/collection1/
[junit4:junit4] 2> 566627 T3084 C629 P52561 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438633647346286592), 3 (1438633647366209536)]} 0 2
[junit4:junit4] 2> 566628 T3084 C629 P52561 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:52561 DONE. sync succeeded
[junit4:junit4] 2> 566628 T3084 C629 P52561 oasc.SolrCore.execute [collection1] webapp= path=/get params={version=2&getVersions=100&wt=javabin&qt=/get&distrib=false&sync=http://127.0.0.1:34080/collection1/} status=0 QTime=11
[junit4:junit4] 2> 566628 T3068 oasc.SyncStrategy.syncToMe http://127.0.0.1:34080/collection1/: sync completed with http://127.0.0.1:52561/collection1/
[junit4:junit4] 2> 566629 T3068 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={core=collection1&version=2&wt=javabin&collection=collection1&shard=shard1&action=REQUESTSYNCSHARD} status=0 QTime=21
[junit4:junit4] 2> 566629 T3100 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={version=2&wt=javabin&collection=collection1&shard=shard1&action=SYNCSHARD} status=0 QTime=27
[junit4:junit4] 2> 566630 T3032 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 566630 T3032 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 566631 T3032 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 566632 T3032 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 566634 T3050 C627 P38912 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 566686 T3050 C627 P38912 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2adb1318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@2adb1318; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 566686 T3050 C627 P38912 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 566687 T3050 C627 P38912 oass.SolrIndexSearcher.<init> Opening Searcher@6957aa83 main
[junit4:junit4] 2> 566688 T3050 C627 P38912 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 566689 T3058 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6957aa83 main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):c4)}
[junit4:junit4] 2> 566689 T3050 C627 P38912 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={version=2&wt=javabin&waitSearcher=true&commit=true&softCommit=false} {commit=} 0 55
[junit4:junit4] 2> 566692 T3118 C625 P36524 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:34080/collection1/, StdNode: http://127.0.0.1:52561/collection1/, StdNode: http://127.0.0.1:58505/collection1/, StdNode: http://127.0.0.1:36524/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 566697 T3070 C626 P34080 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 566697 T3102 C628 P58505 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 566699 T3120 C625 P36524 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 566699 T3086 C629 P52561 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 566716 T3070 C626 P34080 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7ab07238; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.MMapDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371987938478/jetty1/index lockFactory=org.apa
[...truncated too long message...]
a is not real, it was synthetically created to match currency.xml. It is modeled after the data format available from openexchangerates.org. See https://openexchangerates.org/documentation for details",
[junit4:junit4] 1> "license": "http://www.apache.org/licenses/LICENSE-2.0",
[junit4:junit4] 1> "timestamp": 1332070464,
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> "IMPORTANT NOTE": "In order for tests to work, this data must be kept in sync with ./currency.xml",
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> "base": "USD",
[junit4:junit4] 1> "rates": {
[junit4:junit4] 1> "USD": 1,
[junit4:junit4] 1> "JPY": 81.29,
[junit4:junit4] 1> "EUR": 2.5,
[junit4:junit4] 1> "GBP": 0.5,
[junit4:junit4] 1> "MXN": 2.0
[junit4:junit4] 1> }
[junit4:junit4] 1> }
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/configs/conf1/solrconfig.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/stopwords.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89914601536225294-127.0.0.1:34080_-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89914601536225290-127.0.0.1:58505_-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89914601536225283-127.0.0.1:38912_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89914601536225292-127.0.0.1:36524_-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89914601536225283-127.0.0.1:38912_-n_0000000000"}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:34080_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:36524_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:58505_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:38912_ (0)
[junit4:junit4] 1> /solr/clusterstate.json (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "collection1":{
[junit4:junit4] 1> "router":"compositeId",
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{
[junit4:junit4] 1> "127.0.0.1:34080__collection1":{
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:34080_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:34080"},
[junit4:junit4] 1> "127.0.0.1:52561__collection1":{
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:52561_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:52561"},
[junit4:junit4] 1> "127.0.0.1:58505__collection1":{
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:58505_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:58505"},
[junit4:junit4] 1> "127.0.0.1:36524__collection1":{
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:36524_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:36524",
[junit4:junit4] 1> "leader":"true"}}}}},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "router":"compositeId",
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"127.0.0.1:38912__collection1":{
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:38912_",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:38912",
[junit4:junit4] 1> "leader":"true"}}}}}}
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=7C221E7B74E4A52 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fr -Dtests.timezone=Asia/Chongqing -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 100s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:34080/collection1lastClient and got 243 from http://127.0.0.1:58505/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([7C221E7B74E4A52:8624AFFFC0112A6E]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2> 644901 T3032 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 100164 T3031 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=fr, timezone=Asia/Chongqing
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic amd64/Oracle Corporation 1.8.0-ea (64-bit)/cpus=8,threads=1,free=274723840,total=360710144
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestReplicationHandler, ShardRoutingTest, TestReloadAndDeleteDocs, TermVectorComponentTest, SolrInfoMBeanTest, QueryElevationComponentTest, TestCloudManagedSchema, TestSerializedLuceneMatchVersion, TestCharFilters, CSVRequestHandlerTest, TestFunctionQuery, QueryParsingTest, TestPropInjectDefaults, SchemaVersionSpecificBehaviorTest, TestPluginEnable, TestSolrIndexConfig, TestLFUCache, DirectUpdateHandlerOptimizeTest, ZkControllerTest, OverseerTest, AliasIntegrationTest, TestSolrQueryParserResource, TestManagedSchema, TestCopyFieldCollectionResource, TestDocumentBuilder, TestValueSourceCache, UpdateRequestProcessorFactoryTest, TestStressVersions, TestPHPSerializedResponseWriter, TermsComponentTest, DirectSolrConnectionTest, JsonLoaderTest, ChaosMonkeyNothingIsSafeTest, AnalysisAfterCoreReloadTest, SliceStateUpdateTest, LukeRequestHandlerTest, RecoveryZkTest, TestSolrCoreProperties, TestTrie, SuggesterTest, PrimUtilsTest, LeaderElectionTest, TestPhraseSuggestions, SimpleFacetsTest, TestRealTimeGet, CachingDirectoryFactoryTest, PeerSyncTest, CircularListTest, PreAnalyzedFieldTest, ResourceLoaderTest, DateFieldTest, LegacyHTMLStripCharFilterTest, TestFastWriter, TestGroupingSearch, SolrTestCaseJ4Test, SolrCoreTest, MBeansHandlerTest, ClusterStateUpdateTest, FieldMutatingUpdateProcessorTest, ZkCLITest, BadComponentTest, TestSort, SuggesterFSTTest, TestDocSet, TestWriterPerf, RAMDirectoryFactoryTest, ExternalFileFieldSortTest, CoreAdminHandlerTest, ShowFileRequestHandlerTest, TestCoreContainer, TestCloudManagedSchemaAddField, TestExtendedDismaxParser, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, SyncSliceTest]
[junit4:junit4] Completed on J0 in 100.69s, 1 test, 1 failure <<< FAILURES!
[...truncated 454 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:392: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:372: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1246: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:890: There were test failures: 298 suites, 1256 tests, 1 failure, 16 ignored (7 assumptions)
Total time: 38 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0-ea-b93 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_25) - Build # 6200
- Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/6200/
Java: 32bit/jdk1.7.0_25 -client -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:60329/collection1lastClient and got 257 from http://127.0.0.1:43420/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:60329/collection1lastClient and got 257 from http://127.0.0.1:43420/collection1
at __randomizedtesting.SeedInfo.seed([D0E059EB6C8C28CC:5106D7F31BD348F0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 9587 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 311150 T1194 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 311155 T1194 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1371992051942
[junit4:junit4] 2> 311156 T1194 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 311157 T1195 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 311259 T1194 oasc.ZkTestServer.run start zk server on port:51451
[junit4:junit4] 2> 311261 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 311366 T1201 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@598bfe name:ZooKeeperConnection Watcher:127.0.0.1:51451 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 311367 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 311367 T1194 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 311375 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 311376 T1203 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c9ee3a name:ZooKeeperConnection Watcher:127.0.0.1:51451/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 311376 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 311377 T1194 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 311382 T1194 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 311384 T1194 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 311385 T1194 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 311387 T1194 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 311388 T1194 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 311392 T1194 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 311392 T1194 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 311397 T1194 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 311398 T1194 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 311400 T1194 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 311401 T1194 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 311403 T1194 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 311404 T1194 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 311416 T1194 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 311417 T1194 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 311429 T1194 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 311430 T1194 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 311433 T1194 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 311433 T1194 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 311435 T1194 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 311436 T1194 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 311539 T1194 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 311550 T1194 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:33097
[junit4:junit4] 2> 311551 T1194 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 311551 T1194 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 311551 T1194 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371992052227
[junit4:junit4] 2> 311552 T1194 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371992052227/solr.xml
[junit4:junit4] 2> 311552 T1194 oasc.CoreContainer.<init> New CoreContainer 31778908
[junit4:junit4] 2> 311553 T1194 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371992052227/'
[junit4:junit4] 2> 311553 T1194 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371992052227/'
[junit4:junit4] 2> 311621 T1194 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 311622 T1194 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 311622 T1194 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 311622 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 311623 T1194 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 311623 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 311623 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 311624 T1194 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 311624 T1194 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 311625 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 311629 T1194 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 311629 T1194 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51451/solr
[junit4:junit4] 2> 311630 T1194 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 311631 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 311632 T1214 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f61a95 name:ZooKeeperConnection Watcher:127.0.0.1:51451 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 311633 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 311635 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 311639 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 311667 T1216 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@198941f name:ZooKeeperConnection Watcher:127.0.0.1:51451/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 311668 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 311696 T1194 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 311700 T1194 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 311703 T1194 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 311704 T1194 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33097_
[junit4:junit4] 2> 311705 T1194 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33097_
[junit4:junit4] 2> 311707 T1194 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 311714 T1194 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 311716 T1194 oasc.Overseer.start Overseer (id=89914871116201987-127.0.0.1:33097_-n_0000000000) starting
[junit4:junit4] 2> 311718 T1194 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 311733 T1218 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 311733 T1194 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 311739 T1194 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 311743 T1194 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 311746 T1217 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 311748 T1219 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 311748 T1219 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 313249 T1217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 313250 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:33097_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:33097"}
[junit4:junit4] 2> 313250 T1217 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 313250 T1217 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 313252 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 313749 T1219 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371992052227/collection1
[junit4:junit4] 2> 313749 T1219 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 313750 T1219 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 313750 T1219 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 313751 T1219 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371992052227/collection1/'
[junit4:junit4] 2> 313752 T1219 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371992052227/collection1/lib/README' to classloader
[junit4:junit4] 2> 313752 T1219 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1371992052227/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 313797 T1219 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 313857 T1219 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 313859 T1219 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 313866 T1219 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 314385 T1219 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 314386 T1219 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 314386 T1219 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 314393 T1219 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 314397 T1219 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 314412 T1219 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 314416 T1219 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 314419 T1219 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 314420 T1219 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 314420 T1219 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 314421 T1219 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 314422 T1219 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 314422 T1219 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 314422 T1219 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 314433 T1219 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1371992052227/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data/
[junit4:junit4] 2> 314433 T1219 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@46692c
[junit4:junit4] 2> 314434 T1219 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 314434 T1219 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data
[junit4:junit4] 2> 314435 T1219 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data/index/
[junit4:junit4] 2> 314435 T1219 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 314436 T1219 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data/index
[junit4:junit4] 2> 314439 T1219 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19fd58a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 314440 T1219 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 314442 T1219 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 314442 T1219 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 314443 T1219 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 314444 T1219 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 314445 T1219 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 314445 T1219 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 314445 T1219 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 314446 T1219 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 314446 T1219 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 314450 T1219 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 314455 T1219 oass.SolrIndexSearcher.<init> Opening Searcher@1d32d8d main
[junit4:junit4] 2> 314455 T1219 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 314456 T1219 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 314459 T1220 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d32d8d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 314462 T1219 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 314462 T1219 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:33097 collection:control_collection shard:shard1
[junit4:junit4] 2> 314462 T1219 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 314467 T1219 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 314474 T1219 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 314474 T1219 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 314474 T1219 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:33097/collection1/
[junit4:junit4] 2> 314474 T1219 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 314475 T1219 oasc.SyncStrategy.syncToMe http://127.0.0.1:33097/collection1/ has no replicas
[junit4:junit4] 2> 314475 T1219 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:33097/collection1/
[junit4:junit4] 2> 314475 T1219 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 314755 T1217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 314764 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 314785 T1219 oasc.ZkController.register We are http://127.0.0.1:33097/collection1/ and leader is http://127.0.0.1:33097/collection1/
[junit4:junit4] 2> 314785 T1219 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33097
[junit4:junit4] 2> 314785 T1219 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 314786 T1219 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 314786 T1219 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 314789 T1219 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 314790 T1194 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 314790 T1194 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 314791 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 314794 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 314795 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 314796 T1223 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19c91cd name:ZooKeeperConnection Watcher:127.0.0.1:51451/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 314796 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 314799 T1194 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 314801 T1194 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 314893 T1194 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 314907 T1194 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:60329
[junit4:junit4] 2> 314908 T1194 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 314909 T1194 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 314909 T1194 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371992055588
[junit4:junit4] 2> 314909 T1194 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371992055588/solr.xml
[junit4:junit4] 2> 314910 T1194 oasc.CoreContainer.<init> New CoreContainer 20415535
[junit4:junit4] 2> 314911 T1194 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371992055588/'
[junit4:junit4] 2> 314911 T1194 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371992055588/'
[junit4:junit4] 2> 314969 T1194 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 314969 T1194 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 314970 T1194 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 314970 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 314970 T1194 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 314971 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 314971 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 314972 T1194 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 314972 T1194 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 314973 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 314977 T1194 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 314978 T1194 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51451/solr
[junit4:junit4] 2> 314979 T1194 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 314987 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 314990 T1234 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@176ce5b name:ZooKeeperConnection Watcher:127.0.0.1:51451 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 314992 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 314994 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 315000 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 315000 T1236 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1e22c0e name:ZooKeeperConnection Watcher:127.0.0.1:51451/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 315002 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 315012 T1194 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 316015 T1194 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60329_
[junit4:junit4] 2> 316016 T1194 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:60329_
[junit4:junit4] 2> 316018 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 316018 T1223 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 316018 T1236 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 316019 T1216 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 316031 T1237 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 316031 T1237 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 316268 T1217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 316269 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:33097__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:33097_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:33097"}
[junit4:junit4] 2> 316271 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60329_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60329"}
[junit4:junit4] 2> 316271 T1217 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 316271 T1217 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 316278 T1236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 316278 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 316278 T1223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 317032 T1237 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371992055588/collection1
[junit4:junit4] 2> 317033 T1237 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 317034 T1237 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 317034 T1237 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 317035 T1237 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371992055588/collection1/'
[junit4:junit4] 2> 317035 T1237 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371992055588/collection1/lib/README' to classloader
[junit4:junit4] 2> 317036 T1237 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1371992055588/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 317075 T1237 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 317125 T1237 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 317127 T1237 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 317134 T1237 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 317684 T1237 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 317685 T1237 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 317686 T1237 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 317694 T1237 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 317699 T1237 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 317712 T1237 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 317716 T1237 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 317720 T1237 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 317721 T1237 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 317721 T1237 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 317721 T1237 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 317722 T1237 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 317722 T1237 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 317722 T1237 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 317722 T1237 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1371992055588/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/
[junit4:junit4] 2> 317723 T1237 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@46692c
[junit4:junit4] 2> 317723 T1237 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 317723 T1237 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1
[junit4:junit4] 2> 317724 T1237 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index/
[junit4:junit4] 2> 317724 T1237 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 317724 T1237 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index
[junit4:junit4] 2> 317728 T1237 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 317728 T1237 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 317730 T1237 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 317731 T1237 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 317731 T1237 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 317732 T1237 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 317732 T1237 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 317733 T1237 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 317733 T1237 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 317734 T1237 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 317734 T1237 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 317737 T1237 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 317741 T1237 oass.SolrIndexSearcher.<init> Opening Searcher@c79662 main
[junit4:junit4] 2> 317742 T1237 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 317742 T1237 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 317755 T1238 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c79662 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 317768 T1237 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 317768 T1237 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:60329 collection:collection1 shard:shard1
[junit4:junit4] 2> 317771 T1237 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 317783 T1237 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 317787 T1237 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 317787 T1237 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 317788 T1237 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60329/collection1/
[junit4:junit4] 2> 317788 T1237 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 317788 T1237 oasc.SyncStrategy.syncToMe http://127.0.0.1:60329/collection1/ has no replicas
[junit4:junit4] 2> 317788 T1237 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:60329/collection1/
[junit4:junit4] 2> 317789 T1237 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 319285 T1217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 319291 T1236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 319291 T1223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 319291 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 319312 T1237 oasc.ZkController.register We are http://127.0.0.1:60329/collection1/ and leader is http://127.0.0.1:60329/collection1/
[junit4:junit4] 2> 319312 T1237 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60329
[junit4:junit4] 2> 319312 T1237 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 319313 T1237 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 319313 T1237 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 319315 T1237 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 319316 T1194 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 319317 T1194 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 319317 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 319417 T1194 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 319420 T1194 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:38090
[junit4:junit4] 2> 319421 T1194 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 319421 T1194 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 319421 T1194 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371992060108
[junit4:junit4] 2> 319422 T1194 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371992060108/solr.xml
[junit4:junit4] 2> 319422 T1194 oasc.CoreContainer.<init> New CoreContainer 27338502
[junit4:junit4] 2> 319424 T1194 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371992060108/'
[junit4:junit4] 2> 319424 T1194 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371992060108/'
[junit4:junit4] 2> 319501 T1194 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 319501 T1194 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 319501 T1194 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 319502 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 319502 T1194 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 319502 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 319503 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 319503 T1194 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 319504 T1194 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 319504 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 319508 T1194 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 319509 T1194 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51451/solr
[junit4:junit4] 2> 319510 T1194 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 319511 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 319515 T1250 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25236b name:ZooKeeperConnection Watcher:127.0.0.1:51451 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 319516 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 319521 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 319527 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 319537 T1252 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13c11ce name:ZooKeeperConnection Watcher:127.0.0.1:51451/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 319537 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 319546 T1194 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 320548 T1194 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38090_
[junit4:junit4] 2> 320550 T1194 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:38090_
[junit4:junit4] 2> 320551 T1223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 320552 T1236 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 320552 T1236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 320552 T1252 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 320552 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 320553 T1223 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 320554 T1216 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 320560 T1253 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 320560 T1253 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 320800 T1217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 320800 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:60329__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:60329_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:60329"}
[junit4:junit4] 2> 320802 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:38090_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:38090"}
[junit4:junit4] 2> 320803 T1217 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 320803 T1217 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 320805 T1223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 320805 T1236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 320805 T1252 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 320807 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 321561 T1253 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371992060108/collection1
[junit4:junit4] 2> 321562 T1253 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 321563 T1253 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 321563 T1253 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 321565 T1253 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371992060108/collection1/'
[junit4:junit4] 2> 321566 T1253 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371992060108/collection1/lib/README' to classloader
[junit4:junit4] 2> 321566 T1253 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1371992060108/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 321615 T1253 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 321666 T1253 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 321669 T1253 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 321675 T1253 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 322163 T1253 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 322164 T1253 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 322164 T1253 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 322174 T1253 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 322179 T1253 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 322193 T1253 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 322196 T1253 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 322203 T1253 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 322205 T1253 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 322205 T1253 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 322206 T1253 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 322206 T1253 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 322206 T1253 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 322207 T1253 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 322207 T1253 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1371992060108/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/
[junit4:junit4] 2> 322207 T1253 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@46692c
[junit4:junit4] 2> 322208 T1253 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 322208 T1253 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2
[junit4:junit4] 2> 322209 T1253 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index/
[junit4:junit4] 2> 322209 T1253 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 322210 T1253 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index
[junit4:junit4] 2> 322214 T1253 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1231ff4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 322215 T1253 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 322216 T1253 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 322217 T1253 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 322217 T1253 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 322218 T1253 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 322219 T1253 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 322219 T1253 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 322220 T1253 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 322221 T1253 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 322221 T1253 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 322225 T1253 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 322228 T1253 oass.SolrIndexSearcher.<init> Opening Searcher@1793040 main
[junit4:junit4] 2> 322229 T1253 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 322229 T1253 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 322235 T1254 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1793040 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 322237 T1253 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 322237 T1253 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:38090 collection:collection1 shard:shard1
[junit4:junit4] 2> 322242 T1253 oasc.ZkController.register We are http://127.0.0.1:38090/collection1/ and leader is http://127.0.0.1:60329/collection1/
[junit4:junit4] 2> 322243 T1253 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:38090
[junit4:junit4] 2> 322243 T1253 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 322243 T1253 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C288 name=collection1 org.apache.solr.core.SolrCore@f8f4e0 url=http://127.0.0.1:38090/collection1 node=127.0.0.1:38090_ C288_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:38090_, base_url=http://127.0.0.1:38090}
[junit4:junit4] 2> 322244 T1255 C288 P38090 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 322245 T1255 C288 P38090 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 322245 T1253 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 322245 T1255 C288 P38090 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 322246 T1255 C288 P38090 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 322247 T1255 C288 P38090 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 322247 T1194 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 322247 T1194 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 322248 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 322252 T1231 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:38090__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 322319 T1217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 322320 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:38090__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:38090_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:38090"}
[junit4:junit4] 2> 322323 T1236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 322323 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 322323 T1252 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 322323 T1223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 322333 T1194 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 322335 T1194 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:43420
[junit4:junit4] 2> 322336 T1194 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 322336 T1194 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 322337 T1194 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371992063038
[junit4:junit4] 2> 322337 T1194 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371992063038/solr.xml
[junit4:junit4] 2> 322338 T1194 oasc.CoreContainer.<init> New CoreContainer 24543588
[junit4:junit4] 2> 322338 T1194 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371992063038/'
[junit4:junit4] 2> 322338 T1194 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371992063038/'
[junit4:junit4] 2> 322397 T1194 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 322398 T1194 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 322398 T1194 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 322399 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 322399 T1194 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 322400 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 322400 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 322401 T1194 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 322401 T1194 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 322402 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 322406 T1194 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 322406 T1194 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51451/solr
[junit4:junit4] 2> 322407 T1194 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 322408 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 322409 T1267 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f06d05 name:ZooKeeperConnection Watcher:127.0.0.1:51451 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 322409 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 322410 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 322417 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 322418 T1269 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a27501 name:ZooKeeperConnection Watcher:127.0.0.1:51451/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 322418 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 322421 T1194 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 323253 T1231 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:38090__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 323253 T1231 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:38090__collection1&state=recovering&nodeName=127.0.0.1:38090_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 323424 T1194 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43420_
[junit4:junit4] 2> 323425 T1194 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:43420_
[junit4:junit4] 2> 323427 T1223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 323427 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 323427 T1236 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 323427 T1252 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 323428 T1236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 323428 T1252 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 323429 T1223 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 323429 T1216 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 323428 T1269 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 323434 T1270 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 323435 T1270 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 323827 T1217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 323827 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43420_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43420"}
[junit4:junit4] 2> 323827 T1217 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 323828 T1217 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 323833 T1252 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 323833 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 323833 T1236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 323833 T1269 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 323833 T1223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 324436 T1270 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371992063038/collection1
[junit4:junit4] 2> 324436 T1270 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 324440 T1270 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 324440 T1270 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 324441 T1270 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371992063038/collection1/'
[junit4:junit4] 2> 324442 T1270 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371992063038/collection1/lib/README' to classloader
[junit4:junit4] 2> 324443 T1270 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1371992063038/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 324489 T1270 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 324533 T1270 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 324534 T1270 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 324538 T1270 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 325044 T1270 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 325044 T1270 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 325045 T1270 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 325055 T1270 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 325059 T1270 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 325070 T1270 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 325073 T1270 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 325077 T1270 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 325078 T1270 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 325078 T1270 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 325078 T1270 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 325079 T1270 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 325079 T1270 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 325079 T1270 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 325080 T1270 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1371992063038/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/
[junit4:junit4] 2> 325080 T1270 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@46692c
[junit4:junit4] 2> 325080 T1270 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 325081 T1270 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3
[junit4:junit4] 2> 325081 T1270 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index/
[junit4:junit4] 2> 325082 T1270 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 325082 T1270 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index
[junit4:junit4] 2> 325085 T1270 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ad4bf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 325086 T1270 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 325089 T1270 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 325089 T1270 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 325089 T1270 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 325090 T1270 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 325091 T1270 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 325091 T1270 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 325091 T1270 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 325092 T1270 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 325092 T1270 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 325095 T1270 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 325098 T1270 oass.SolrIndexSearcher.<init> Opening Searcher@154fe52 main
[junit4:junit4] 2> 325099 T1270 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 325099 T1270 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 325106 T1271 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@154fe52 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 325108 T1270 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 325108 T1270 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:43420 collection:collection1 shard:shard1
[junit4:junit4] 2> 325110 T1270 oasc.ZkController.register We are http://127.0.0.1:43420/collection1/ and leader is http://127.0.0.1:60329/collection1/
[junit4:junit4] 2> 325110 T1270 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:43420
[junit4:junit4] 2> 325111 T1270 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 325111 T1270 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C289 name=collection1 org.apache.solr.core.SolrCore@458105 url=http://127.0.0.1:43420/collection1 node=127.0.0.1:43420_ C289_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:43420_, base_url=http://127.0.0.1:43420}
[junit4:junit4] 2> 325112 T1272 C289 P43420 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 325112 T1270 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 325113 T1272 C289 P43420 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 325116 T1194 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 325119 T1194 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 325119 T1272 C289 P43420 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 325120 T1272 C289 P43420 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 325120 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 325121 T1272 C289 P43420 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 325125 T1231 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:43420__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 325202 T1194 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 325211 T1194 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52605
[junit4:junit4] 2> 325212 T1194 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 325212 T1194 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 325213 T1194 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371992065910
[junit4:junit4] 2> 325213 T1194 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371992065910/solr.xml
[junit4:junit4] 2> 325213 T1194 oasc.CoreContainer.<init> New CoreContainer 2517383
[junit4:junit4] 2> 325215 T1194 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371992065910/'
[junit4:junit4] 2> 325215 T1194 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371992065910/'
[junit4:junit4] 2> ASYNC NEW_CORE C290 name=collection1 org.apache.solr.core.SolrCore@f8f4e0 url=http://127.0.0.1:38090/collection1 node=127.0.0.1:38090_ C290_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:38090_, base_url=http://127.0.0.1:38090}
[junit4:junit4] 2> 325254 T1255 C290 P38090 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:60329/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 325254 T1255 C290 P38090 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38090 START replicas=[http://127.0.0.1:60329/collection1/] nUpdates=100
[junit4:junit4] 2> 325255 T1255 C290 P38090 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 325256 T1255 C290 P38090 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 325256 T1255 C290 P38090 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 325264 T1255 C290 P38090 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 325264 T1255 C290 P38090 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 325264 T1255 C290 P38090 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:60329/collection1/. core=collection1
[junit4:junit4] 2> 325264 T1255 C290 P38090 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C291 name=collection1 org.apache.solr.core.SolrCore@8dad81 url=http://127.0.0.1:60329/collection1 node=127.0.0.1:60329_ C291_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:60329_, base_url=http://127.0.0.1:60329, leader=true}
[junit4:junit4] 2> 325275 T1230 C291 P60329 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 325275 T1228 C291 P60329 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 325277 T1194 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 325277 T1194 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 325278 T1194 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 325277 T1230 C291 P60329 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 325278 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 325278 T1230 C291 P60329 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 325279 T1194 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 325280 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 325280 T1194 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 325280 T1194 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 325281 T1194 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 325281 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 325285 T1230 C291 P60329 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 325289 T1230 C291 P60329 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 325290 T1230 C291 P60329 oass.SolrIndexSearcher.<init> Opening Searcher@1afee5d realtime
[junit4:junit4] 2> 325291 T1230 C291 P60329 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 325291 T1230 C291 P60329 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 16
[junit4:junit4] 2> 325292 T1255 C290 P38090 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 325292 T1255 C290 P38090 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 325300 T1228 C291 P60329 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 325301 T1228 C291 P60329 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=7
[junit4:junit4] 2> 325301 T1255 C290 P38090 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 325302 T1255 C290 P38090 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 325302 T1194 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 325302 T1194 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51451/solr
[junit4:junit4] 2> 325303 T1194 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 325303 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 325302 T1255 C290 P38090 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 325308 T1229 C291 P60329 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 325308 T1285 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@48f551 name:ZooKeeperConnection Watcher:127.0.0.1:51451 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 325308 T1255 C290 P38090 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 325309 T1255 C290 P38090 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index.20130623095426095
[junit4:junit4] 2> 325310 T1255 C290 P38090 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index.20130623095426095 lockFactory=org.apache.lucene.store.NativeFSLockFactory@12235fa; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 325311 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 325315 T1230 C291 P60329 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 325316 T1255 C290 P38090 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 325315 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 325326 T1255 C290 P38090 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 325326 T1255 C290 P38090 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 325328 T1255 C290 P38090 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1231ff4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1231ff4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 325330 T1255 C290 P38090 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 325330 T1255 C290 P38090 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 325330 T1194 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 325331 T1255 C290 P38090 oass.SolrIndexSearcher.<init> Opening Searcher@1c1ab17 main
[junit4:junit4] 2> 325332 T1254 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c1ab17 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 325332 T1288 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10f9a60 name:ZooKeeperConnection Watcher:127.0.0.1:51451/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 325333 T1194 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 325333 T1255 C290 P38090 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index.20130623095426095 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index.20130623095426095;done=true>>]
[junit4:junit4] 2> 325333 T1255 C290 P38090 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index.20130623095426095
[junit4:junit4] 2> 325334 T1255 C290 P38090 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index.20130623095426095
[junit4:junit4] 2> 325334 T1255 C290 P38090 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 325335 T1255 C290 P38090 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 325335 T1255 C290 P38090 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 325335 T1255 C290 P38090 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 325336 T1255 C290 P38090 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 325337 T1194 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 325347 T1217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 325348 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:43420__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43420_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43420"}
[junit4:junit4] 2> 325351 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:38090__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:38090_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:38090"}
[junit4:junit4] 2> 325375 T1236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 325375 T1223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 325376 T1269 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 325375 T1252 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 325378 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 325375 T1288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 326126 T1231 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:43420__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 326127 T1231 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:43420__collection1&state=recovering&nodeName=127.0.0.1:43420_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1002
[junit4:junit4] 2> 326348 T1194 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52605_
[junit4:junit4] 2> 326349 T1194 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52605_
[junit4:junit4] 2> 326351 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 326351 T1223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 326351 T1269 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 326351 T1252 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 326351 T1236 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 326352 T1236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 326353 T1216 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 326352 T1252 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 326352 T1288 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 326353 T1288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 326353 T1223 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 326354 T1269 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 326358 T1289 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 326359 T1289 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 326885 T1217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 326886 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:52605_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52605"}
[junit4:junit4] 2> 326886 T1217 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 326886 T1217 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 326889 T1288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 326889 T1236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 326889 T1252 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 326889 T1269 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 326890 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 326890 T1223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 327360 T1289 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371992065910/collection1
[junit4:junit4] 2> 327360 T1289 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 327361 T1289 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 327361 T1289 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 327362 T1289 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371992065910/collection1/'
[junit4:junit4] 2> 327363 T1289 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371992065910/collection1/lib/README' to classloader
[junit4:junit4] 2> 327363 T1289 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1371992065910/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 327411 T1289 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_44
[junit4:junit4] 2> 327458 T1289 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 327460 T1289 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 327465 T1289 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 327940 T1289 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 327940 T1289 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
[junit4:junit4] 2> 327941 T1289 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 327949 T1289 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 327962 T1289 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 327973 T1289 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 327978 T1289 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 327982 T1289 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 327983 T1289 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 327984 T1289 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 327984 T1289 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 327987 T1289 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 327988 T1289 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 327988 T1289 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 327989 T1289 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1371992065910/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/
[junit4:junit4] 2> 327989 T1289 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@46692c
[junit4:junit4] 2> 327991 T1289 oasc.SolrCore.initDirectoryFactory solr.NRTCachingDirectoryFactory
[junit4:junit4] 2> 327991 T1289 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4
[junit4:junit4] 2> 327992 T1289 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index/
[junit4:junit4] 2> 327992 T1289 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 327993 T1289 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index
[junit4:junit4] 2> 327997 T1289 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a10089; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 327998 T1289 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 328001 T1289 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 328002 T1289 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 328003 T1289 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 328004 T1289 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 328005 T1289 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 328005 T1289 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 328005 T1289 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 328006 T1289 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 328006 T1289 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 328010 T1289 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 328015 T1289 oass.SolrIndexSearcher.<init> Opening Searcher@65355e main
[junit4:junit4] 2> 328016 T1289 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 328016 T1289 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 328021 T1290 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@65355e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 328023 T1289 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 328023 T1289 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52605 collection:collection1 shard:shard1
[junit4:junit4] 2> 328025 T1289 oasc.ZkController.register We are http://127.0.0.1:52605/collection1/ and leader is http://127.0.0.1:60329/collection1/
[junit4:junit4] 2> 328025 T1289 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52605
[junit4:junit4] 2> 328025 T1289 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 328025 T1289 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C292 name=collection1 org.apache.solr.core.SolrCore@9688fa url=http://127.0.0.1:52605/collection1 node=127.0.0.1:52605_ C292_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:52605_, base_url=http://127.0.0.1:52605}
[junit4:junit4] 2> 328026 T1291 C292 P52605 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 328026 T1291 C292 P52605 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 328026 T1291 C292 P52605 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 328027 T1291 C292 P52605 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 328027 T1289 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 328027 T1291 C292 P52605 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 328029 T1194 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 2> 328029 T1194 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 328030 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 328033 T1231 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:52605__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 328033 T1194 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 328034 T1194 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 328034 T1194 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 328035 T1194 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C293 name=collection1 org.apache.solr.core.SolrCore@458105 url=http://127.0.0.1:43420/collection1 node=127.0.0.1:43420_ C293_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:43420_, base_url=http://127.0.0.1:43420}
[junit4:junit4] 2> 328128 T1272 C293 P43420 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:60329/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 328128 T1272 C293 P43420 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:43420 START replicas=[http://127.0.0.1:60329/collection1/] nUpdates=100
[junit4:junit4] 2> 328129 T1272 C293 P43420 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 328129 T1272 C293 P43420 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 328129 T1272 C293 P43420 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 328130 T1272 C293 P43420 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 328130 T1272 C293 P43420 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 328130 T1272 C293 P43420 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:60329/collection1/. core=collection1
[junit4:junit4] 2> 328130 T1272 C293 P43420 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C294 name=collection1 org.apache.solr.core.SolrCore@8dad81 url=http://127.0.0.1:60329/collection1 node=127.0.0.1:60329_ C294_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:60329_, base_url=http://127.0.0.1:60329, leader=true}
[junit4:junit4] 2> 328131 T1229 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 328134 T1230 C294 P60329 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 328145 T1230 C294 P60329 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 328146 T1230 C294 P60329 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 328146 T1230 C294 P60329 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 328147 T1230 C294 P60329 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 14
[junit4:junit4] 2> 328148 T1272 C293 P43420 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 328148 T1272 C293 P43420 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 328150 T1229 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 328151 T1272 C293 P43420 oash.SnapPuller.fetchLatestIndex Master's generation: 3
[junit4:junit4] 2> 328151 T1272 C293 P43420 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 328151 T1272 C293 P43420 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 328153 T1230 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=3&version=2} status=0 QTime=0
[junit4:junit4] 2> 328154 T1272 C293 P43420 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 328155 T1272 C293 P43420 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index.20130623095428940
[junit4:junit4] 2> 328155 T1272 C293 P43420 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index.20130623095428940 lockFactory=org.apache.lucene.store.NativeFSLockFactory@42f759; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 328158 T1228 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_3&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=3} status=0 QTime=0
[junit4:junit4] 2> 328159 T1272 C293 P43420 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 328163 T1272 C293 P43420 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 328163 T1272 C293 P43420 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 328165 T1272 C293 P43420 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ad4bf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ad4bf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 328166 T1272 C293 P43420 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 328168 T1272 C293 P43420 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 328169 T1272 C293 P43420 oass.SolrIndexSearcher.<init> Opening Searcher@918c6c main
[junit4:junit4] 2> 328170 T1271 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@918c6c main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 328171 T1272 C293 P43420 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index.20130623095428940 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index.20130623095428940;done=true>>]
[junit4:junit4] 2> 328171 T1272 C293 P43420 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index.20130623095428940
[junit4:junit4] 2> 328173 T1272 C293 P43420 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index.20130623095428940
[junit4:junit4] 2> 328173 T1272 C293 P43420 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 328174 T1272 C293 P43420 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 328174 T1272 C293 P43420 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 328174 T1272 C293 P43420 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 328175 T1272 C293 P43420 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 328394 T1217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 328394 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:52605__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:52605_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52605"}
[junit4:junit4] 2> 328397 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:43420__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:43420_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:43420"}
[junit4:junit4] 2> 328415 T1223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 328415 T1269 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 328415 T1236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 328415 T1252 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 328415 T1288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 328415 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 329033 T1231 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:52605__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 329034 T1231 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:52605__collection1&state=recovering&nodeName=127.0.0.1:52605_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 2> 329036 T1194 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 330037 T1194 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C292_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:52605_, base_url=http://127.0.0.1:52605}
[junit4:junit4] 2> 331036 T1291 C292 P52605 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:60329/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 331036 T1291 C292 P52605 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:52605 START replicas=[http://127.0.0.1:60329/collection1/] nUpdates=100
[junit4:junit4] 2> 331037 T1291 C292 P52605 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 331037 T1291 C292 P52605 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 331037 T1291 C292 P52605 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 331038 T1291 C292 P52605 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 331038 T1291 C292 P52605 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 331038 T1291 C292 P52605 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:60329/collection1/. core=collection1
[junit4:junit4] 2> 331038 T1291 C292 P52605 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 331039 T1194 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 331043 T1229 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 331046 T1228 C294 P60329 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 331055 T1228 C294 P60329 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 331056 T1228 C294 P60329 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 331056 T1228 C294 P60329 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 331057 T1228 C294 P60329 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 11
[junit4:junit4] 2> 331057 T1291 C292 P52605 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 331057 T1291 C292 P52605 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 331059 T1229 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 331060 T1291 C292 P52605 oash.SnapPuller.fetchLatestIndex Master's generation: 4
[junit4:junit4] 2> 331060 T1291 C292 P52605 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 331060 T1291 C292 P52605 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 331062 T1230 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=4&version=2} status=0 QTime=0
[junit4:junit4] 2> 331062 T1291 C292 P52605 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 331063 T1291 C292 P52605 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index.20130623095431849
[junit4:junit4] 2> 331064 T1291 C292 P52605 oash.SnapPuller.fetchLatestIndex Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index.20130623095431849 lockFactory=org.apache.lucene.store.NativeFSLockFactory@a1c59b; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 2> 331066 T1228 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_4&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=4} status=0 QTime=0
[junit4:junit4] 2> 331068 T1291 C292 P52605 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 331080 T1291 C292 P52605 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 331080 T1291 C292 P52605 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 331082 T1291 C292 P52605 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a10089; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a10089; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 331084 T1291 C292 P52605 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 331084 T1291 C292 P52605 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 331084 T1291 C292 P52605 oass.SolrIndexSearcher.<init> Opening Searcher@11d8812 main
[junit4:junit4] 2> 331086 T1290 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@11d8812 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 331087 T1291 C292 P52605 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index.20130623095431849 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index.20130623095431849;done=true>>]
[junit4:junit4] 2> 331087 T1291 C292 P52605 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index.20130623095431849
[junit4:junit4] 2> 331087 T1291 C292 P52605 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index.20130623095431849
[junit4:junit4] 2> 331087 T1291 C292 P52605 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 331088 T1291 C292 P52605 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 331088 T1291 C292 P52605 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 331088 T1291 C292 P52605 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 331089 T1291 C292 P52605 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 331422 T1217 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 331423 T1217 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:52605__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:52605_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:52605"}
[junit4:junit4] 2> 331425 T1223 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 331426 T1269 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 331426 T1288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 331426 T1236 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 331425 T1252 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 331425 T1216 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 332040 T1194 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 332046 T1194 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C295 name=collection1 org.apache.solr.core.SolrCore@3ab283 url=http://127.0.0.1:33097/collection1 node=127.0.0.1:33097_ C295_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:33097_, base_url=http://127.0.0.1:33097, leader=true}
[junit4:junit4] 2> 332053 T1209 C295 P33097 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 332057 T1209 C295 P33097 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19fd58a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 332058 T1209 C295 P33097 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 332085 T1209 C295 P33097 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19fd58a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19fd58a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 332085 T1209 C295 P33097 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 332086 T1209 C295 P33097 oass.SolrIndexSearcher.<init> Opening Searcher@bedbd3 main
[junit4:junit4] 2> 332086 T1209 C295 P33097 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 332088 T1220 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@bedbd3 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 332089 T1209 C295 P33097 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 36
[junit4:junit4] 2> ASYNC NEW_CORE C296 name=collection1 org.apache.solr.core.SolrCore@f8f4e0 url=http://127.0.0.1:38090/collection1 node=127.0.0.1:38090_ C296_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:38090_, base_url=http://127.0.0.1:38090}
[junit4:junit4] 2> 332101 T1248 C296 P38090 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:60329/collection1/, StdNode: http://127.0.0.1:38090/collection1/, StdNode: http://127.0.0.1:43420/collection1/, StdNode: http://127.0.0.1:52605/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 332115 T1228 C294 P60329 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 332127 T1246 C296 P38090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C293_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:43420_, base_url=http://127.0.0.1:43420}
[junit4:junit4] 2> 332133 T1263 C293 P43420 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 332135 T1228 C294 P60329 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=4
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 332135 T1228 C294 P60329 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 332136 T1228 C294 P60329 oass.SolrIndexSearcher.<init> Opening Searcher@9008e8 main
[junit4:junit4] 2> 332137 T1228 C294 P60329 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 332138 T1238 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9008e8 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 332138 T1228 C294 P60329 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 23
[junit4:junit4] 2> C292_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:52605_, base_url=http://127.0.0.1:52605}
[junit4:junit4] 2> 332142 T1279 C292 P52605 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 332145 T1246 C296 P38090 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1231ff4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1231ff4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 332145 T1246 C296 P38090 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 332150 T1246 C296 P38090 oass.SolrIndexSearcher.<init> Opening Searcher@1df2817 main
[junit4:junit4] 2> 332151 T1246 C296 P38090 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 332152 T1254 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1df2817 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 332153 T1246 C296 P38090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 26
[junit4:junit4] 2> 332165 T1263 C293 P43420 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ad4bf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ad4bf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 332165 T1263 C293 P43420 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 332166 T1263 C293 P43420 oass.SolrIndexSearcher.<init> Opening Searcher@1baaf2 main
[junit4:junit4] 2> 332167 T1263 C293 P43420 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 332168 T1271 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1baaf2 main{StandardDirectoryReader(segments_3:1:nrt)}
[junit4:junit4] 2> 332168 T1263 C293 P43420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 35
[junit4:junit4] 2> 332175 T1279 C292 P52605 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a10089; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a10089; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 332175 T1279 C292 P52605 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 332177 T1279 C292 P52605 oass.SolrIndexSearcher.<init> Opening Searcher@18f5241 main
[junit4:junit4] 2> 332177 T1279 C292 P52605 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 332179 T1290 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@18f5241 main{StandardDirectoryReader(segments_4:1:nrt)}
[junit4:junit4] 2> 332180 T1279 C292 P52605 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 38
[junit4:junit4] 2> 332180 T1248 C296 P38090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 80
[junit4:junit4] 2> 332181 T1194 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 332183 T1229 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 332185 T1247 C296 P38090 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 332186 T1264 C293 P43420 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 332189 T1280 C292 P52605 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 334193 T1212 C295 P33097 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1438637962012131328)} 0 1
[junit4:junit4] 2> 334202 T1279 C292 P52605 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438637962017374208&update.from=http://127.0.0.1:60329/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438637962017374208)} 0 1
[junit4:junit4] 2> 334202 T1245 C296 P38090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438637962017374208&update.from=http://127.0.0.1:60329/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438637962017374208)} 0 1
[junit4:junit4] 2> 334209 T1265 C293 P43420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1438637962017374208&update.from=http://127.0.0.1:60329/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1438637962017374208)} 0 6
[junit4:junit4] 2> 334210 T1231 C294 P60329 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1438637962017374208)} 0 13
[junit4:junit4] 2> 334210 T1248 C296 P38090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 15
[junit4:junit4] 2> 334214 T1211 C295 P33097 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1438637962034151424)]} 0 1
[junit4:junit4] 2> 334220 T1246 C296 P38090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60329/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438637962037297152)]} 0 1
[junit4:junit4] 2> 334220 T1280 C292 P52605 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60329/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438637962037297152)]} 0 1
[junit4:junit4] 2> 334222 T1262 C293 P43420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60329/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1438637962037297152)]} 0 1
[junit4:junit4] 2> 334223 T1229 C294 P60329 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1438637962037297152)]} 0 8
[junit4:junit4] 2> 334226 T1210 C295 P33097 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1438637962046734336)]} 0 1
[junit4:junit4] 2> 334232 T1247 C296 P38090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60329/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438637962050928640)]} 0 1
[junit4:junit4] 2> 334232 T1264 C293 P43420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60329/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438637962050928640)]} 0 1
[junit4:junit4] 2> 334232 T1281 C292 P52605 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60329/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1438637962050928640)]} 0 1
[junit4:junit4] 2> 334233 T1228 C294 P60329 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43420/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1438637962050928640)]} 0 4
[junit4:junit4] 2> 334233 T1263 C293 P43420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1]} 0 6
[junit4:junit4] 2> 334236 T1209 C295 P33097 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1438637962057220096)]} 0 1
[junit4:junit4] 2> 334240 T1231 C294 P60329 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:38090/collection1/]
[junit4:junit4] 2> 334241 T1231 C294 P60329 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:38090/collection1/ against:[http://127.0.0.1:38090/collection1/] result:true
[junit4:junit4] 2> 334241 T1231 C294 P60329 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:43420/collection1/ against:[http://127.0.0.1:38090/collection1/] result:false
[junit4:junit4] 2> 334241 T1231 C294 P60329 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:52605/collection1/ against:[http://127.0.0.1:38090/collection1/] result:false
[junit4:junit4] 2> 334245 T1265 C293 P43420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60329/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:38090/collection1/&wt=javabin&version=2} {add=[2 (1438637962063511552)]} 0 1
[junit4:junit4] 2> 334248 T1282 C292 P52605 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60329/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:38090/collection1/&wt=javabin&version=2} {add=[2 (1438637962063511552)]} 0 1
[junit4:junit4] 2> 334249 T1231 C294 P60329 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:38090/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:38090/collection1/&wt=javabin&version=2} {add=[2 (1438637962063511552)]} 0 9
[junit4:junit4] 2> 334251 T1245 C296 P38090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={test.distrib.skip.servers=http://127.0.0.1:38090/collection1/&wt=javabin&version=2} {add=[2]} 0 13
[junit4:junit4] 2> 334255 T1212 C295 P33097 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1438637962077143040)]} 0 1
[junit4:junit4] 2> 334259 T1229 C294 P60329 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:38090/collection1/]
[junit4:junit4] 2> 334259 T1229 C294 P60329 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:38090/collection1/ against:[http://127.0.0.1:38090/collection1/] result:true
[junit4:junit4] 2> 334259 T1229 C294 P60329 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:43420/collection1/ against:[http://127.0.0.1:38090/collection1/] result:false
[junit4:junit4] 2> 334260 T1229 C294 P60329 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:52605/collection1/ against:[http://127.0.0.1:38090/collection1/] result:false
[junit4:junit4] 2> 334263 T1264 C293 P43420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60329/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:38090/collection1/&wt=javabin&version=2} {add=[3 (1438637962083434496)]} 0 1
[junit4:junit4] 2> 334263 T1279 C292 P52605 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:60329/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:38090/collection1/&wt=javabin&version=2} {add=[3 (1438637962083434496)]} 0 1
[junit4:junit4] 2> 334264 T1229 C294 P60329 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:43420/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:38090/collection1/&wt=javabin&version=2} {add=[3 (1438637962083434496)]} 0 5
[junit4:junit4] 2> 334264 T1262 C293 P43420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={test.distrib.skip.servers=http://127.0.0.1:38090/collection1/&test.distrib.skip.servers=http://127.0.0.1:43420/collection1/&wt=javabin&version=2} {add=[3]} 0 7
[junit4:junit4] 2> 334266 T1211 C295 P33097 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 334305 T1211 C295 P33097 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19fd58a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19fd58a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> 334306 T1211 C295 P33097 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 334309 T1211 C295 P33097 oass.SolrIndexSearcher.<init> Opening Searcher@fed4e7 main
[junit4:junit4] 2> 334310 T1211 C295 P33097 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 334311 T1220 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@fed4e7 main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 334312 T1211 C295 P33097 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 46
[junit4:junit4] 2> 334314 T1248 C296 P38090 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:60329/collection1/, StdNode: http://127.0.0.1:38090/collection1/, StdNode: http://127.0.0.1:43420/collection1/, StdNode: http://127.0.0.1:52605/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 334316 T1230 C294 P60329 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 334317 T1263 C293 P43420 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 334318 T1280 C292 P52605 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 334317 T1246 C296 P38090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 334365 T1280 C292 P52605 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a10089; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a10089; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 334366 T1280 C292 P52605 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 334368 T1280 C292 P52605 oass.SolrIndexSearcher.<init> Opening Searcher@17962cd main
[junit4:junit4] 2> 334368 T1280 C292 P52605 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 334370 T1290 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@17962cd main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 334370 T1280 C292 P52605 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 52
[junit4:junit4] 2> 334375 T1246 C296 P38090 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1231ff4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1231ff4; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 334376 T1246 C296 P38090 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 334375 T1263 C293 P43420 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ad4bf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ad4bf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> 334377 T1263 C293 P43420 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 5
[junit4:junit4] 2> 334378 T1246 C296 P38090 oass.SolrIndexSearcher.<init> Opening Searcher@55f4a2 main
[junit4:junit4] 2> 334379 T1246 C296 P38090 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 334380 T1254 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@55f4a2 main{StandardDirectoryReader(segments_4:3:nrt _0(4.4):C2)}
[junit4:junit4] 2> 334381 T1246 C296 P38090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 64
[junit4:junit4] 2> 334382 T1263 C293 P43420 oass.SolrIndexSearcher.<init> Opening Searcher@1dc3f0b main
[junit4:junit4] 2> 334383 T1263 C293 P43420 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 334384 T1271 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1dc3f0b main{StandardDirectoryReader(segments_5:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 334384 T1263 C293 P43420 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 67
[junit4:junit4] 2> 334405 T1230 C294 P60329 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 334406 T1230 C294 P60329 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 334408 T1230 C294 P60329 oass.SolrIndexSearcher.<init> Opening Searcher@68a49b main
[junit4:junit4] 2> 334409 T1230 C294 P60329 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 334410 T1238 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68a49b main{StandardDirectoryReader(segments_6:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 334410 T1230 C294 P60329 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 94
[junit4:junit4] 2> 334411 T1248 C296 P38090 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 97
[junit4:junit4] 2> 334411 T1194 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):230
[junit4:junit4] 2> 334412 T1194 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 334413 T1194 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 334415 T1228 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 334418 T1247 C296 P38090 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=1
[junit4:junit4] 2> 334420 T1265 C293 P43420 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 334422 T1281 C292 P52605 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 334423 T1194 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 334431 T1264 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 334431 T1264 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 334435 T1231 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 334436 T1231 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:60329/collection1/
[junit4:junit4] 2> 334436 T1231 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:60329 START replicas=[http://127.0.0.1:38090/collection1/, http://127.0.0.1:43420/collection1/, http://127.0.0.1:52605/collection1/] nUpdates=100
[junit4:junit4] 2> 334438 T1246 C296 P38090 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 334439 T1263 C293 P43420 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 334439 T1231 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60329 Received 3 versions from 127.0.0.1:38090/collection1/
[junit4:junit4] 2> 334439 T1282 C292 P52605 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 334440 T1231 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60329 Our versions are newer. ourLowThreshold=1438637962017374208 otherHigh=1438637962050928640
[junit4:junit4] 2> 334440 T1231 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60329 Received 5 versions from 127.0.0.1:43420/collection1/
[junit4:junit4] 2> 334441 T1231 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60329 Our versions are newer. ourLowThreshold=1438637962017374208 otherHigh=1438637962063511552
[junit4:junit4] 2> 334441 T1231 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60329 Received 5 versions from 127.0.0.1:52605/collection1/
[junit4:junit4] 2> 334441 T1231 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:60329 Our versions are newer. ourLowThreshold=1438637962017374208 otherHigh=1438637962063511552
[junit4:junit4] 2> 334442 T1231 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:60329 DONE. sync succeeded
[junit4:junit4] 2> 334442 T1231 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 334443 T1231 oasc.SyncStrategy.syncToMe http://127.0.0.1:60329/collection1/: try and ask http://127.0.0.1:38090/collection1/ to sync
[junit4:junit4] 2> 334444 T1231 oasc.SyncStrategy.syncToMe http://127.0.0.1:60329/collection1/: try and ask http://127.0.0.1:43420/collection1/ to sync
[junit4:junit4] 2> 334445 T1231 oasc.SyncStrategy.syncToMe http://127.0.0.1:60329/collection1/: try and ask http://127.0.0.1:52605/collection1/ to sync
[junit4:junit4] 2> 334445 T1247 C296 P38090 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38090 START replicas=[http://127.0.0.1:60329/collection1/] nUpdates=100
[junit4:junit4] 2> 334447 T1265 C293 P43420 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:43420 START replicas=[http://127.0.0.1:60329/collection1/] nUpdates=100
[junit4:junit4] 2> 334448 T1279 C292 P52605 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:52605 START replicas=[http://127.0.0.1:60329/collection1/] nUpdates=100
[junit4:junit4] 2> 334448 T1229 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 334449 T1229 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 334450 T1247 C296 P38090 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:38090 Received 5 versions from 127.0.0.1:60329/collection1/
[junit4:junit4] 2> 334449 T1265 C293 P43420 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:43420 Received 5 versions from 127.0.0.1:60329/collection1/
[junit4:junit4] 2> 334450 T1247 C296 P38090 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:38090 Requesting updates from 127.0.0.1:60329/collection1/n=2 versions=[1438637962083434496, 1438637962063511552]
[junit4:junit4] 2> 334450 T1265 C293 P43420 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:43420 Our versions are newer. ourLowThreshold=1438637962017374208 otherHigh=1438637962063511552
[junit4:junit4] 2> 334451 T1265 C293 P43420 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:43420 DONE. sync succeeded
[junit4:junit4] 2> 334451 T1265 C293 P43420 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:60329/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=4
[junit4:junit4] 2> 334452 T1231 oasc.SyncStrategy.syncToMe http://127.0.0.1:60329/collection1/: sync completed with http://127.0.0.1:43420/collection1/
[junit4:junit4] 2> 334453 T1228 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 334453 T1230 C294 P60329 oasc.SolrCore.execute [collection1] webapp= path=/get params={distrib=false&getUpdates=1438637962083434496,1438637962063511552&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 334453 T1279 C292 P52605 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:52605 Received 5 versions from 127.0.0.1:60329/collection1/
[junit4:junit4] 2> 334454 T1279 C292 P52605 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:52605 Our versions are newer. ourLowThreshold=1438637962017374208 otherHigh=1438637962063511552
[junit4:junit4] 2> 334455 T1279 C292 P52605 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:52605 DONE. sync succeeded
[junit4:junit4] 2> 334455 T1279 C292 P52605 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:60329/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=7
[junit4:junit4] 2> 334455 T1231 oasc.SyncStrategy.syncToMe http://127.0.0.1:60329/collection1/: sync completed with http://127.0.0.1:52605/collection1/
[junit4:junit4] 2> 334457 T1247 C296 P38090 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1438637962063511552), 3 (1438637962083434496)]} 0 3
[junit4:junit4] 2> 334457 T1247 C296 P38090 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:38090 DONE. sync succeeded
[junit4:junit4] 2> 334457 T1247 C296 P38090 oasc.SolrCore.execute [collection1] webapp= path=/get params={sync=http://127.0.0.1:60329/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=12
[junit4:junit4] 2> 334458 T1231 oasc.SyncStrategy.syncToMe http://127.0.0.1:60329/collection1/: sync completed with http://127.0.0.1:38090/collection1/
[junit4:junit4] 2> 334458 T1231 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=23
[junit4:junit4] 2> 334459 T1264 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=28
[junit4:junit4] 2> 334459 T1194 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 334459 T1194 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 334460 T1194 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 334461 T1194 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 334463 T1210 C295 P33097 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 334485 T1210 C295 P33097 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19fd58a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@19fd58a; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> 334485 T1210 C295 P33097 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 4
[junit4:junit4] 2> 334486 T1210 C295 P33097 oass.SolrIndexSearcher.<init> Opening Searcher@8bba9b main
[junit4:junit4] 2> 334487 T1210 C295 P33097 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 334490 T1220 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@8bba9b main{StandardDirectoryReader(segments_3:3:nrt _0(4.4):C4)}
[junit4:junit4] 2> 334491 T1210 C295 P33097 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 28
[junit4:junit4] 2> 334493 T1280 C292 P52605 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:60329/collection1/, StdNode: http://127.0.0.1:38090/collection1/, StdNode: http://127.0.0.1:43420/collection1/, StdNode: http://127.0.0.1:52605/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 334504 T1229 C294 P60329 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 334505 T1281 C292 P52605 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 334506 T1263 C293 P43420 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 334506 T1246 C296 P38090 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 334546 T1281 C292 P52605 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a10089; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@a10089; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_7,generation=7}
[junit4:junit4] 2> 334547 T1281 C292 P52605 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 334547 T1263 C293 P43420 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ad4bf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_5,generation=5}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ad4bf3; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> 334547 T1229 C294 P60329 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=5
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_4,generation=4}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_6,generation=6}
[junit4:junit4] 2> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-1371992051942/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@7b521f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_7,generation=7}
[junit4:junit4] 2> 334549 T1229 C294 P60329 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 7
[junit4:junit4] 2> 334549 T1263 C293 P43420 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 6
[junit4:junit4] 2> 334548 T1281 C292 P52605 oass.SolrIndexSearcher.<init> Opening Searcher@9d31a0 main
[junit4:junit4] 2> 334550 T1229 C294 P60329 oass.SolrIndexSearcher.<init> Opening Searcher@15130d6 main
[junit4:junit4] 2> 334552 T1263 C293 P43420 oass.SolrIndexSearcher.<init> Opening Searcher@4c107d m
[...truncated too long message...]
:38090_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:38090"},
[junit4:junit4] 1> "127.0.0.1:43420__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:43420_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:43420",
[junit4:junit4] 1> "leader":"true"},
[junit4:junit4] 1> "127.0.0.1:52605__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:52605_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:52605"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"127.0.0.1:33097__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "node_name":"127.0.0.1:33097_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:33097",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/live_nodes (4)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:52605_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:43420_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:60329_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:33097_ (0)
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89914871116201998-127.0.0.1:60329__collection1-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89914871116201996-127.0.0.1:52605__collection1-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89914871116201994-127.0.0.1:43420__collection1-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:43420_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:43420"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89914871116201987-127.0.0.1:33097__collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:33097_",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:33097"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89914871116201987-127.0.0.1:33097_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89914871116201998-127.0.0.1:60329_-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89914871116201996-127.0.0.1:52605_-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89914871116201994-127.0.0.1:43420_-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89914871116201987-127.0.0.1:33097_-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=D0E059EB6C8C28CC -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_EC -Dtests.timezone=America/Argentina/ComodRivadavia -Dtests.file.encoding=US-ASCII
[junit4:junit4] FAILURE 100s J0 | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:60329/collection1lastClient and got 257 from http://127.0.0.1:43420/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([D0E059EB6C8C28CC:5106D7F31BD348F0]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1018)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:238)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2> 411216 T1194 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 100068 T1193 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> 412377 T1288 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 412377 T1288 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 412378 T1288 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Appending, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=es_EC, timezone=America/Argentina/ComodRivadavia
[junit4:junit4] 2> NOTE: Linux 3.2.0-48-generic i386/Oracle Corporation 1.7.0_25 (32-bit)/cpus=8,threads=1,free=46681192,total=210960384
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestReloadAndDeleteDocs, SOLR749Test, SliceStateUpdateTest, TestWriterPerf, CircularListTest, ScriptEngineTest, PrimitiveFieldTypeTest, SystemInfoHandlerTest, TestSolrQueryParser, SoftAutoCommitTest, SolrCoreTest, TestDocSet, TestFuzzyAnalyzedSuggestions, DirectUpdateHandlerOptimizeTest, SpatialFilterTest, TestCharFilters, TestPHPSerializedResponseWriter, ZkCLITest, TestJoin, AliasIntegrationTest, OverseerTest, CachingDirectoryFactoryTest, TestSolrIndexConfig, TestCollationKeyRangeQueries, ShowFileRequestHandlerTest, PrimUtilsTest, AlternateDirectoryTest, BadComponentTest, PluginInfoTest, RecoveryZkTest, TermsComponentTest, TestRealTimeGet, AnalysisAfterCoreReloadTest, LoggingHandlerTest, TestRemoteStreaming, TestLMDirichletSimilarityFactory, SuggesterTest, ZkControllerTest, LukeRequestHandlerTest, TestLRUCache, SyncSliceTest]
[junit4:junit4] Completed on J0 in 101.36s, 1 test, 1 failure <<< FAILURES!
[...truncated 591 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:392: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:372: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1246: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:890: There were test failures: 298 suites, 1256 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 36 minutes 21 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_25 -client -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure