You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/06/28 12:46:46 UTC
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 604 -
Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/604/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
1 tests failed.
REGRESSION: org.apache.solr.cloud.SyncSliceTest.testDistribSearch
Error Message:
shard1 is not consistent. Got 305 from http://127.0.0.1:50863/x_avr/bb/collection1lastClient and got 248 from http://127.0.0.1:50870/x_avr/bb/collection1
Stack Trace:
java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:50863/x_avr/bb/collection1lastClient and got 248 from http://127.0.0.1:50870/x_avr/bb/collection1
at __randomizedtesting.SeedInfo.seed([60679E00F3D4D777:E1811018848BB74B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1045)
at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:237)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 9580 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.SyncSliceTest
[junit4:junit4] 2> 1781265 T5474 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /x_avr/bb
[junit4:junit4] 2> 1781312 T5474 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-SyncSliceTest-1372414566269
[junit4:junit4] 2> 1781315 T5474 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1781316 T5475 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1781417 T5474 oasc.ZkTestServer.run start zk server on port:50856
[junit4:junit4] 2> 1781419 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1781428 T5481 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f88d7f8 name:ZooKeeperConnection Watcher:127.0.0.1:50856 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1781428 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1781429 T5474 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1781440 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1781444 T5483 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@55ebe322 name:ZooKeeperConnection Watcher:127.0.0.1:50856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1781444 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1781445 T5474 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1781452 T5474 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1781459 T5474 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1781464 T5474 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1781472 T5474 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1781473 T5474 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1781487 T5474 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1781488 T5474 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1781499 T5474 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1781501 T5474 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1781507 T5474 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1781508 T5474 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1781515 T5474 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1781516 T5474 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1781522 T5474 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1781523 T5474 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1781530 T5474 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1781531 T5474 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1781538 T5474 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1781538 T5474 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1781545 T5474 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1781546 T5474 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1781958 T5474 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1781981 T5474 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50859
[junit4:junit4] 2> 1781983 T5474 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1781984 T5474 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1781985 T5474 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372414566510
[junit4:junit4] 2> 1781986 T5474 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372414566510/solr.xml
[junit4:junit4] 2> 1781986 T5474 oasc.CoreContainer.<init> New CoreContainer 1713762612
[junit4:junit4] 2> 1781988 T5474 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372414566510/'
[junit4:junit4] 2> 1781989 T5474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372414566510/'
[junit4:junit4] 2> 1782123 T5474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1782124 T5474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1782125 T5474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1782125 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1782126 T5474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1782127 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1782128 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1782129 T5474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1782130 T5474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1782131 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1782157 T5474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1782158 T5474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50856/solr
[junit4:junit4] 2> 1782159 T5474 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1782161 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1782165 T5494 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@409fce51 name:ZooKeeperConnection Watcher:127.0.0.1:50856 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1782166 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1782170 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1782186 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1782191 T5496 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16212d95 name:ZooKeeperConnection Watcher:127.0.0.1:50856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1782191 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1782195 T5474 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1782204 T5474 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1782213 T5474 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1782217 T5474 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50859_x_avr%2Fbb
[junit4:junit4] 2> 1782220 T5474 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50859_x_avr%2Fbb
[junit4:junit4] 2> 1782227 T5474 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1782239 T5474 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1782244 T5474 oasc.Overseer.start Overseer (id=89942561015595011-127.0.0.1:50859_x_avr%2Fbb-n_0000000000) starting
[junit4:junit4] 2> 1782252 T5474 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1782262 T5498 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1782263 T5474 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1782268 T5474 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1782272 T5474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1782279 T5497 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1782286 T5499 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1782286 T5499 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1782289 T5499 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 1783794 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1783795 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50859_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50859/x_avr/bb"}
[junit4:junit4] 2> 1783796 T5497 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 1783796 T5497 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1783804 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1784292 T5499 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 1784292 T5499 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372414566510/collection1
[junit4:junit4] 2> 1784292 T5499 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 1784294 T5499 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1784295 T5499 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 1784298 T5499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372414566510/collection1/'
[junit4:junit4] 2> 1784300 T5499 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1372414566510/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1784301 T5499 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-controljetty-1372414566510/collection1/lib/README' to classloader
[junit4:junit4] 2> 1784367 T5499 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1784443 T5499 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1784447 T5499 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1784453 T5499 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1785299 T5499 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1785313 T5499 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1785318 T5499 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1785349 T5499 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1785356 T5499 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1785363 T5499 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1785371 T5499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1785371 T5499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1785371 T5499 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1785374 T5499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1785374 T5499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1785375 T5499 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1785375 T5499 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-controljetty-1372414566510/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372414566268/control/data/
[junit4:junit4] 2> 1785375 T5499 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5a04648b
[junit4:junit4] 2> 1785377 T5499 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1372414566268/control/data
[junit4:junit4] 2> 1785377 T5499 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372414566268/control/data/index/
[junit4:junit4] 2> 1785377 T5499 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372414566268/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1785378 T5499 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1372414566268/control/data/index
[junit4:junit4] 2> 1785383 T5499 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b6ffa80 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2474b185),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1785383 T5499 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1785387 T5499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1785387 T5499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1785388 T5499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1785389 T5499 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1785390 T5499 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1785390 T5499 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1785390 T5499 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1785391 T5499 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1785392 T5499 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1785406 T5499 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1785415 T5499 oass.SolrIndexSearcher.<init> Opening Searcher@64023199 main
[junit4:junit4] 2> 1785417 T5499 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1785418 T5499 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1785422 T5500 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@64023199 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1785424 T5499 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1785424 T5499 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50859/x_avr/bb collection:control_collection shard:shard1
[junit4:junit4] 2> 1785426 T5499 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1785442 T5499 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 1785449 T5499 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1785449 T5499 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1785450 T5499 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50859/x_avr/bb/collection1/
[junit4:junit4] 2> 1785450 T5499 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1785450 T5499 oasc.SyncStrategy.syncToMe http://127.0.0.1:50859/x_avr/bb/collection1/ has no replicas
[junit4:junit4] 2> 1785451 T5499 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50859/x_avr/bb/collection1/ shard1
[junit4:junit4] 2> 1785451 T5499 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 1786819 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1786840 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1786867 T5499 oasc.ZkController.register We are http://127.0.0.1:50859/x_avr/bb/collection1/ and leader is http://127.0.0.1:50859/x_avr/bb/collection1/
[junit4:junit4] 2> 1786868 T5499 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50859/x_avr/bb
[junit4:junit4] 2> 1786868 T5499 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1786869 T5499 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1786869 T5499 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1786872 T5499 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1786875 T5474 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1786876 T5474 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1786878 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1786900 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1786905 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1786911 T5503 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a5f6ad5 name:ZooKeeperConnection Watcher:127.0.0.1:50856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1786911 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1786914 T5474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1786920 T5474 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1787358 T5474 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1787363 T5474 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50863
[junit4:junit4] 2> 1787365 T5474 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1787365 T5474 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1787366 T5474 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372414571875
[junit4:junit4] 2> 1787367 T5474 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372414571875/solr.xml
[junit4:junit4] 2> 1787367 T5474 oasc.CoreContainer.<init> New CoreContainer 271335508
[junit4:junit4] 2> 1787368 T5474 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372414571875/'
[junit4:junit4] 2> 1787368 T5474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372414571875/'
[junit4:junit4] 2> 1787516 T5474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1787517 T5474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1787518 T5474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1787518 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1787519 T5474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1787520 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1787520 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1787521 T5474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1787521 T5474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1787522 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1787541 T5474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1787542 T5474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50856/solr
[junit4:junit4] 2> 1787542 T5474 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1787544 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1787548 T5514 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ab71b4e name:ZooKeeperConnection Watcher:127.0.0.1:50856 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1787549 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1787553 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1787569 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1787579 T5516 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@195d4817 name:ZooKeeperConnection Watcher:127.0.0.1:50856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1787579 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1787588 T5474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1788351 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1788353 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50859_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50859/x_avr/bb"}
[junit4:junit4] 2> 1788362 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1788362 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1788362 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1788599 T5474 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50863_x_avr%2Fbb
[junit4:junit4] 2> 1788603 T5474 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50863_x_avr%2Fbb
[junit4:junit4] 2> 1788609 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1788609 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 1788611 T5516 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1788612 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1788616 T5496 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1788618 T5503 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1788634 T5517 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1788635 T5517 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1788638 T5517 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 1789877 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1789879 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50863_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50863/x_avr/bb"}
[junit4:junit4] 2> 1789879 T5497 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 1789880 T5497 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1789891 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1789892 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1789891 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1790640 T5517 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 1790640 T5517 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372414571875/collection1
[junit4:junit4] 2> 1790641 T5517 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1790642 T5517 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1790642 T5517 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1790645 T5517 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372414571875/collection1/'
[junit4:junit4] 2> 1790647 T5517 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1372414571875/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1790648 T5517 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty1-1372414571875/collection1/lib/README' to classloader
[junit4:junit4] 2> 1790722 T5517 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1790817 T5517 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1790820 T5517 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1790828 T5517 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1791797 T5517 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1791813 T5517 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1791819 T5517 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1791856 T5517 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1791865 T5517 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1791875 T5517 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1791878 T5517 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1791879 T5517 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1791880 T5517 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1791883 T5517 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1791884 T5517 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1791884 T5517 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1791885 T5517 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty1-1372414571875/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty1/
[junit4:junit4] 2> 1791885 T5517 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5a04648b
[junit4:junit4] 2> 1791886 T5517 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty1
[junit4:junit4] 2> 1791887 T5517 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty1/index/
[junit4:junit4] 2> 1791887 T5517 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1791887 T5517 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty1/index
[junit4:junit4] 2> 1791892 T5517 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4bcd36c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e431790),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1791892 T5517 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1791897 T5517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1791897 T5517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1791898 T5517 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1791900 T5517 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1791902 T5517 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1791902 T5517 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1791903 T5517 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1791904 T5517 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1791905 T5517 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1791921 T5517 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1791930 T5517 oass.SolrIndexSearcher.<init> Opening Searcher@4783cd81 main
[junit4:junit4] 2> 1791932 T5517 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1791933 T5517 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1791940 T5518 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@4783cd81 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1791945 T5517 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1791945 T5517 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50863/x_avr/bb collection:collection1 shard:shard1
[junit4:junit4] 2> 1791947 T5517 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 1791969 T5517 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 1791976 T5517 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1791976 T5517 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1791977 T5517 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50863/x_avr/bb/collection1/
[junit4:junit4] 2> 1791977 T5517 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1791977 T5517 oasc.SyncStrategy.syncToMe http://127.0.0.1:50863/x_avr/bb/collection1/ has no replicas
[junit4:junit4] 2> 1791978 T5517 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50863/x_avr/bb/collection1/ shard1
[junit4:junit4] 2> 1791978 T5517 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 1792912 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1792929 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1792930 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1792930 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1792936 T5517 oasc.ZkController.register We are http://127.0.0.1:50863/x_avr/bb/collection1/ and leader is http://127.0.0.1:50863/x_avr/bb/collection1/
[junit4:junit4] 2> 1792937 T5517 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50863/x_avr/bb
[junit4:junit4] 2> 1792937 T5517 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1792937 T5517 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1792938 T5517 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1792941 T5517 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1792946 T5474 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1792948 T5474 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1792949 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1793423 T5474 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1793429 T5474 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50866
[junit4:junit4] 2> 1793432 T5474 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1793433 T5474 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1793434 T5474 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372414577933
[junit4:junit4] 2> 1793435 T5474 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372414577933/solr.xml
[junit4:junit4] 2> 1793435 T5474 oasc.CoreContainer.<init> New CoreContainer 1724140342
[junit4:junit4] 2> 1793436 T5474 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372414577933/'
[junit4:junit4] 2> 1793437 T5474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372414577933/'
[junit4:junit4] 2> 1793637 T5474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1793638 T5474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1793638 T5474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1793639 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1793640 T5474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1793640 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1793641 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1793642 T5474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1793642 T5474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1793643 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1793664 T5474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1793665 T5474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50856/solr
[junit4:junit4] 2> 1793666 T5474 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1793668 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1793674 T5530 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7ec99b97 name:ZooKeeperConnection Watcher:127.0.0.1:50856 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1793675 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1793680 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1793699 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1793704 T5532 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72af9904 name:ZooKeeperConnection Watcher:127.0.0.1:50856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1793705 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1793717 T5474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1794452 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1794454 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50863_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50863/x_avr/bb"}
[junit4:junit4] 2> 1794467 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1794467 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1794467 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1794467 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1794729 T5474 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50866_x_avr%2Fbb
[junit4:junit4] 2> 1794733 T5474 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50866_x_avr%2Fbb
[junit4:junit4] 2> 1794739 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1794740 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1794740 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 1794742 T5516 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1794743 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1794749 T5532 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1794751 T5496 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1794752 T5503 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1794770 T5533 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1794770 T5533 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1794773 T5533 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 1795984 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1795986 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50866_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50866/x_avr/bb"}
[junit4:junit4] 2> 1795986 T5497 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 1795987 T5497 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1796001 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1796002 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1796002 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1796003 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1796777 T5533 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 1796778 T5533 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372414577933/collection1
[junit4:junit4] 2> 1796778 T5533 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1796780 T5533 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1796781 T5533 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1796785 T5533 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372414577933/collection1/'
[junit4:junit4] 2> 1796786 T5533 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1372414577933/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1796787 T5533 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty2-1372414577933/collection1/lib/README' to classloader
[junit4:junit4] 2> 1796866 T5533 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1796957 T5533 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1796960 T5533 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1796967 T5533 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1797935 T5533 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1797959 T5533 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1797966 T5533 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1798003 T5533 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1798017 T5533 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1798032 T5533 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1798037 T5533 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1798038 T5533 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1798039 T5533 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1798046 T5533 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1798047 T5533 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1798048 T5533 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1798049 T5533 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty2-1372414577933/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty2/
[junit4:junit4] 2> 1798053 T5533 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5a04648b
[junit4:junit4] 2> 1798055 T5533 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty2
[junit4:junit4] 2> 1798056 T5533 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty2/index/
[junit4:junit4] 2> 1798056 T5533 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1798057 T5533 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty2/index
[junit4:junit4] 2> 1798069 T5533 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a0cdad5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@26516c43),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1798069 T5533 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1798082 T5533 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1798083 T5533 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1798084 T5533 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1798087 T5533 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1798091 T5533 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1798092 T5533 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1798093 T5533 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1798094 T5533 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1798095 T5533 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1798119 T5533 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1798132 T5533 oass.SolrIndexSearcher.<init> Opening Searcher@7db3b95 main
[junit4:junit4] 2> 1798135 T5533 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1798136 T5533 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1798146 T5534 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7db3b95 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1798151 T5533 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1798151 T5533 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50866/x_avr/bb collection:collection1 shard:shard1
[junit4:junit4] 2> 1798161 T5533 oasc.ZkController.register We are http://127.0.0.1:50866/x_avr/bb/collection1/ and leader is http://127.0.0.1:50863/x_avr/bb/collection1/
[junit4:junit4] 2> 1798161 T5533 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50866/x_avr/bb
[junit4:junit4] 2> 1798162 T5533 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1798162 T5533 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C728 name=collection1 org.apache.solr.core.SolrCore@53539c88 url=http://127.0.0.1:50866/x_avr/bb/collection1 node=127.0.0.1:50866_x_avr%2Fbb C728_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:50866_x_avr%2Fbb, base_url=http://127.0.0.1:50866/x_avr/bb}
[junit4:junit4] 2> 1798164 T5535 C728 P50866 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1798165 T5535 C728 P50866 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1798165 T5535 C728 P50866 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1798166 T5535 C728 P50866 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1798165 T5533 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1798169 T5535 C728 P50866 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1798171 T5474 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1798172 T5474 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1798173 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1798207 T5507 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1798670 T5474 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1798677 T5474 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50870
[junit4:junit4] 2> 1798680 T5474 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1798681 T5474 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1798681 T5474 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372414583156
[junit4:junit4] 2> 1798682 T5474 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372414583156/solr.xml
[junit4:junit4] 2> 1798682 T5474 oasc.CoreContainer.<init> New CoreContainer 632182667
[junit4:junit4] 2> 1798683 T5474 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372414583156/'
[junit4:junit4] 2> 1798684 T5474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372414583156/'
[junit4:junit4] 2> 1798833 T5474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1798834 T5474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1798834 T5474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1798835 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1798835 T5474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1798836 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1798836 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1798837 T5474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1798837 T5474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1798838 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1798857 T5474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1798858 T5474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50856/solr
[junit4:junit4] 2> 1798859 T5474 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1798860 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1798866 T5547 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b077db7 name:ZooKeeperConnection Watcher:127.0.0.1:50856 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1798868 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1798873 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1798892 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1798896 T5549 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76ba3c96 name:ZooKeeperConnection Watcher:127.0.0.1:50856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1798897 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1798907 T5474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1799025 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1799027 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50866_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50866/x_avr/bb"}
[junit4:junit4] 2> 1799039 T5549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1799040 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1799040 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1799039 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1799039 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1799209 T5507 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1799211 T5507 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node2&state=recovering&nodeName=127.0.0.1:50866_x_avr%252Fbb&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1004
[junit4:junit4] 2> 1799924 T5474 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50870_x_avr%2Fbb
[junit4:junit4] 2> 1799930 T5474 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50870_x_avr%2Fbb
[junit4:junit4] 2> 1799936 T5549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1799936 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1799937 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1799937 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 1799939 T5516 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1799940 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1799945 T5549 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1799945 T5503 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1799947 T5532 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1799947 T5496 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1799960 T5550 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1799961 T5550 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1799966 T5550 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 1800560 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1800562 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50870_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50870/x_avr/bb"}
[junit4:junit4] 2> 1800562 T5497 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 1800563 T5497 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1800574 T5549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1800574 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1800574 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1800575 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1800575 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1800972 T5550 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 1800973 T5550 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372414583156/collection1
[junit4:junit4] 2> 1800973 T5550 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1800975 T5550 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1800975 T5550 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1800980 T5550 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372414583156/collection1/'
[junit4:junit4] 2> 1800981 T5550 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1372414583156/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1800982 T5550 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty3-1372414583156/collection1/lib/README' to classloader
[junit4:junit4] 2> 1801072 T5550 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1801197 T5550 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1801201 T5550 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1801212 T5550 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> C728_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:50866_x_avr%2Fbb, base_url=http://127.0.0.1:50866/x_avr/bb}
[junit4:junit4] 2> 1801213 T5535 C728 P50866 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50863/x_avr/bb/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1801214 T5535 C728 P50866 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50866/x_avr/bb START replicas=[http://127.0.0.1:50863/x_avr/bb/collection1/] nUpdates=100
[junit4:junit4] 2> 1801215 T5535 C728 P50866 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1801219 T5535 C728 P50866 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1801230 T5535 C728 P50866 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1801230 T5535 C728 P50866 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1801231 T5535 C728 P50866 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1801231 T5535 C728 P50866 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50863/x_avr/bb/collection1/. core=collection1
[junit4:junit4] 2> 1801232 T5535 C728 P50866 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C729 name=collection1 org.apache.solr.core.SolrCore@590d2c5f url=http://127.0.0.1:50863/x_avr/bb/collection1 node=127.0.0.1:50863_x_avr%2Fbb C729_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:50863_x_avr%2Fbb, base_url=http://127.0.0.1:50863/x_avr/bb, leader=true}
[junit4:junit4] 2> 1801233 T5507 C729 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1801263 T5511 C729 P50863 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1801275 T5511 C729 P50863 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4bcd36c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e431790),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1801277 T5511 C729 P50863 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1801277 T5511 C729 P50863 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 1801278 T5511 C729 P50863 oass.SolrIndexSearcher.<init> Opening Searcher@45e0dcf4 realtime
[junit4:junit4] 2> 1801281 T5511 C729 P50863 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1801282 T5511 C729 P50863 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 20
[junit4:junit4] 2> 1801286 T5535 C728 P50866 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1801288 T5535 C728 P50866 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1801296 T5508 C729 P50863 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1801297 T5508 C729 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=3
[junit4:junit4] 2> 1801298 T5535 C728 P50866 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1801299 T5535 C728 P50866 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1801299 T5535 C728 P50866 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1801299 T5535 C728 P50866 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1801302 T5535 C728 P50866 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1802092 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1802094 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50866_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50866/x_avr/bb"}
[junit4:junit4] 2> 1802114 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1802115 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1802114 T5549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1802115 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1802115 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1802554 T5550 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1802569 T5550 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1802575 T5550 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1802630 T5550 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1802638 T5550 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1802650 T5550 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1802654 T5550 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1802656 T5550 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1802656 T5550 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1802659 T5550 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1802659 T5550 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1802660 T5550 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1802660 T5550 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty3-1372414583156/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty3/
[junit4:junit4] 2> 1802661 T5550 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5a04648b
[junit4:junit4] 2> 1802662 T5550 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty3
[junit4:junit4] 2> 1802666 T5550 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty3/index/
[junit4:junit4] 2> 1802668 T5550 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1802668 T5550 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty3/index
[junit4:junit4] 2> 1802673 T5550 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5941403d lockFactory=org.apache.lucene.store.NativeFSLockFactory@e8473b4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1802674 T5550 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1802685 T5550 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1802686 T5550 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1802688 T5550 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1802690 T5550 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1802692 T5550 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1802692 T5550 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1802695 T5550 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1802697 T5550 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1802698 T5550 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1802724 T5550 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1802736 T5550 oass.SolrIndexSearcher.<init> Opening Searcher@7d78ff18 main
[junit4:junit4] 2> 1802738 T5550 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1802739 T5550 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1802752 T5552 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7d78ff18 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1802762 T5550 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1802763 T5550 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50870/x_avr/bb collection:collection1 shard:shard1
[junit4:junit4] 2> 1802774 T5550 oasc.ZkController.register We are http://127.0.0.1:50870/x_avr/bb/collection1/ and leader is http://127.0.0.1:50863/x_avr/bb/collection1/
[junit4:junit4] 2> 1802775 T5550 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50870/x_avr/bb
[junit4:junit4] 2> 1802776 T5550 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1802777 T5550 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C730 name=collection1 org.apache.solr.core.SolrCore@3cc2566c url=http://127.0.0.1:50870/x_avr/bb/collection1 node=127.0.0.1:50870_x_avr%2Fbb C730_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:50870_x_avr%2Fbb, base_url=http://127.0.0.1:50870/x_avr/bb}
[junit4:junit4] 2> 1802779 T5553 C730 P50870 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1802779 T5553 C730 P50870 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1802780 T5553 C730 P50870 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1802780 T5553 C730 P50870 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1802781 T5550 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1802786 T5553 C730 P50870 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1802814 T5474 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1802816 T5474 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1802817 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1802824 T5511 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1803373 T5474 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1803382 T5474 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50877
[junit4:junit4] 2> 1803385 T5474 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1803385 T5474 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1803386 T5474 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372414587795
[junit4:junit4] 2> 1803387 T5474 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372414587795/solr.xml
[junit4:junit4] 2> 1803388 T5474 oasc.CoreContainer.<init> New CoreContainer 447102684
[junit4:junit4] 2> 1803390 T5474 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372414587795/'
[junit4:junit4] 2> 1803391 T5474 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372414587795/'
[junit4:junit4] 2> 1803650 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1803652 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50870_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50870/x_avr/bb"}
[junit4:junit4] 2> 1803667 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1803668 T5549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1803669 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1803668 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1803669 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1803745 T5474 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 1803747 T5474 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1803748 T5474 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 1803749 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1803750 T5474 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1803751 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1803752 T5474 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1803753 T5474 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1803754 T5474 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1803755 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 1803786 T5474 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1803788 T5474 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50856/solr
[junit4:junit4] 2> 1803790 T5474 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1803795 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1803801 T5565 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e28a6f2 name:ZooKeeperConnection Watcher:127.0.0.1:50856 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1803803 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1803810 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1803827 T5511 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1803828 T5511 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:50870_x_avr%252Fbb&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1003
[junit4:junit4] 2> 1803840 T5474 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1803846 T5567 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a2afed5 name:ZooKeeperConnection Watcher:127.0.0.1:50856/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1803846 T5474 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1803861 T5474 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1804873 T5474 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50877_x_avr%2Fbb
[junit4:junit4] 2> 1804876 T5474 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50877_x_avr%2Fbb
[junit4:junit4] 2> 1804882 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1804883 T5549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1804883 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1804883 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 1804887 T5567 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1804887 T5516 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1804888 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1804891 T5503 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1804894 T5532 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1804894 T5496 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1804895 T5549 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1804911 T5568 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1804912 T5568 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1804915 T5568 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 1805196 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1805198 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50877_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50877/x_avr/bb"}
[junit4:junit4] 2> 1805198 T5497 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 1805199 T5497 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 1805209 T5567 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1805209 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1805210 T5549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1805210 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1805209 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1805210 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> C730_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:50870_x_avr%2Fbb, base_url=http://127.0.0.1:50870/x_avr/bb}
[junit4:junit4] 2> 1805840 T5553 C730 P50870 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50863/x_avr/bb/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1805841 T5553 C730 P50870 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50870/x_avr/bb START replicas=[http://127.0.0.1:50863/x_avr/bb/collection1/] nUpdates=100
[junit4:junit4] 2> 1805842 T5553 C730 P50870 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1805843 T5553 C730 P50870 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1805843 T5553 C730 P50870 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1805843 T5553 C730 P50870 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1805843 T5553 C730 P50870 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1805844 T5553 C730 P50870 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50863/x_avr/bb/collection1/. core=collection1
[junit4:junit4] 2> 1805844 T5553 C730 P50870 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C731 name=collection1 org.apache.solr.core.SolrCore@590d2c5f url=http://127.0.0.1:50863/x_avr/bb/collection1 node=127.0.0.1:50863_x_avr%2Fbb C731_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:50863_x_avr%2Fbb, base_url=http://127.0.0.1:50863/x_avr/bb, leader=true}
[junit4:junit4] 2> 1805848 T5511 C731 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 1805872 T5507 C731 P50863 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1805873 T5507 C731 P50863 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 1805875 T5507 C731 P50863 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1805876 T5507 C731 P50863 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 4
[junit4:junit4] 2> 1805878 T5553 C730 P50870 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1805878 T5553 C730 P50870 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1805883 T5512 C731 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 1805884 T5553 C730 P50870 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1805885 T5553 C730 P50870 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1805885 T5553 C730 P50870 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1805886 T5553 C730 P50870 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1805890 T5553 C730 P50870 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1805918 T5568 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 1805919 T5568 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372414587795/collection1
[junit4:junit4] 2> 1805919 T5568 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 1805921 T5568 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1805922 T5568 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 1805926 T5568 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372414587795/collection1/'
[junit4:junit4] 2> 1805928 T5568 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1372414587795/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 1805929 T5568 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.SyncSliceTest-jetty4-1372414587795/collection1/lib/README' to classloader
[junit4:junit4] 2> 1806014 T5568 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1806095 T5568 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1806098 T5568 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1806110 T5568 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 1806726 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1806728 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50870_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50870/x_avr/bb"}
[junit4:junit4] 2> 1806740 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1806741 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1806741 T5549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1806741 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1806741 T5567 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1806741 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1807093 T5568 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1807109 T5568 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1807115 T5568 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1807154 T5568 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1807164 T5568 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1807172 T5568 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1807176 T5568 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1807177 T5568 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1807178 T5568 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1807181 T5568 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1807181 T5568 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1807182 T5568 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1807182 T5568 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.SyncSliceTest-jetty4-1372414587795/collection1/, dataDir=./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty4/
[junit4:junit4] 2> 1807182 T5568 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5a04648b
[junit4:junit4] 2> 1807184 T5568 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty4
[junit4:junit4] 2> 1807184 T5568 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty4/index/
[junit4:junit4] 2> 1807185 T5568 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1807185 T5568 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty4/index
[junit4:junit4] 2> 1807190 T5568 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2c5554a3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2b36df82),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1807191 T5568 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1807197 T5568 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1807198 T5568 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1807199 T5568 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1807200 T5568 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1807201 T5568 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1807202 T5568 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1807204 T5568 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1807205 T5568 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1807206 T5568 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1807223 T5568 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1807232 T5568 oass.SolrIndexSearcher.<init> Opening Searcher@51b203a1 main
[junit4:junit4] 2> 1807234 T5568 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1807234 T5568 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1807242 T5570 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@51b203a1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1807247 T5568 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 1807247 T5568 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:50877/x_avr/bb collection:collection1 shard:shard1
[junit4:junit4] 2> 1807255 T5568 oasc.ZkController.register We are http://127.0.0.1:50877/x_avr/bb/collection1/ and leader is http://127.0.0.1:50863/x_avr/bb/collection1/
[junit4:junit4] 2> 1807256 T5568 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:50877/x_avr/bb
[junit4:junit4] 2> 1807256 T5568 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 1807257 T5568 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C732 name=collection1 org.apache.solr.core.SolrCore@11e2dbfa url=http://127.0.0.1:50877/x_avr/bb/collection1 node=127.0.0.1:50877_x_avr%2Fbb C732_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:50877_x_avr%2Fbb, base_url=http://127.0.0.1:50877/x_avr/bb}
[junit4:junit4] 2> 1807258 T5571 C732 P50877 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 1807259 T5571 C732 P50877 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1807259 T5568 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1807259 T5571 C732 P50877 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 1807260 T5571 C732 P50877 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1807263 T5474 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1807265 T5474 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1807264 T5571 C732 P50877 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1807266 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1807290 T5474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1807291 T5507 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1807293 T5474 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 30 for each attempt
[junit4:junit4] 2> 1807293 T5474 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30
[junit4:junit4] 2> 1807295 T5474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1808266 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1808268 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50877_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50877/x_avr/bb"}
[junit4:junit4] 2> 1808279 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1808279 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1808279 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1808280 T5549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1808279 T5567 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1808281 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1808300 T5507 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 1808301 T5507 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:50877_x_avr%252Fbb&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1010
[junit4:junit4] 2> 1808302 T5474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1809307 T5474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C732_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:50877_x_avr%2Fbb, base_url=http://127.0.0.1:50877/x_avr/bb}
[junit4:junit4] 2> 1810308 T5571 C732 P50877 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50863/x_avr/bb/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1810308 T5571 C732 P50877 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50877/x_avr/bb START replicas=[http://127.0.0.1:50863/x_avr/bb/collection1/] nUpdates=100
[junit4:junit4] 2> 1810310 T5571 C732 P50877 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1810310 T5571 C732 P50877 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 1810311 T5571 C732 P50877 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 1810311 T5571 C732 P50877 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 1810312 T5571 C732 P50877 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1810312 T5571 C732 P50877 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50863/x_avr/bb/collection1/. core=collection1
[junit4:junit4] 2> 1810313 T5571 C732 P50877 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C733 name=collection1 org.apache.solr.core.SolrCore@590d2c5f url=http://127.0.0.1:50863/x_avr/bb/collection1 node=127.0.0.1:50863_x_avr%2Fbb C733_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:50863_x_avr%2Fbb, base_url=http://127.0.0.1:50863/x_avr/bb, leader=true}
[junit4:junit4] 2> 1810313 T5507 C733 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1810315 T5474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1810341 T5511 C733 P50863 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1810343 T5511 C733 P50863 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 1810343 T5511 C733 P50863 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1810344 T5511 C733 P50863 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 3
[junit4:junit4] 2> 1810346 T5571 C732 P50877 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1810347 T5571 C732 P50877 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1810350 T5509 C733 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 1810352 T5571 C732 P50877 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 1810353 T5571 C732 P50877 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 1810353 T5571 C732 P50877 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 1810354 T5571 C732 P50877 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1810357 T5571 C732 P50877 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 1811304 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1811306 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50877_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50877/x_avr/bb"}
[junit4:junit4] 2> 1811321 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1811322 T5549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1811322 T5567 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1811322 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1811324 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1811324 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1811326 T5474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1811331 T5474 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C734 name=collection1 org.apache.solr.core.SolrCore@68f5c3ef url=http://127.0.0.1:50859/x_avr/bb/collection1 node=127.0.0.1:50859_x_avr%2Fbb C734_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:50859_x_avr%2Fbb, base_url=http://127.0.0.1:50859/x_avr/bb, leader=true}
[junit4:junit4] 2> 1811356 T5492 C734 P50859 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1811363 T5492 C734 P50859 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b6ffa80 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2474b185),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1811366 T5492 C734 P50859 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1811367 T5492 C734 P50859 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 1811368 T5492 C734 P50859 oass.SolrIndexSearcher.<init> Opening Searcher@728960e9 main
[junit4:junit4] 2> 1811369 T5492 C734 P50859 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1811372 T5500 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@728960e9 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 1811374 T5492 C734 P50859 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4] 2> ASYNC NEW_CORE C735 name=collection1 org.apache.solr.core.SolrCore@3cc2566c url=http://127.0.0.1:50870/x_avr/bb/collection1 node=127.0.0.1:50870_x_avr%2Fbb C735_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:50870_x_avr%2Fbb, base_url=http://127.0.0.1:50870/x_avr/bb}
[junit4:junit4] 2> 1811419 T5540 C735 P50870 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:50863/x_avr/bb/collection1/, StdNode: http://127.0.0.1:50866/x_avr/bb/collection1/, StdNode: http://127.0.0.1:50870/x_avr/bb/collection1/, StdNode: http://127.0.0.1:50877/x_avr/bb/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 1811439 T5508 C733 P50863 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1811440 T5508 C733 P50863 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 1811442 T5508 C733 P50863 oass.SolrIndexSearcher.<init> Opening Searcher@450da6b4 main
[junit4:junit4] 2> 1811443 T5508 C733 P50863 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1811446 T5518 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@450da6b4 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> ASYNC NEW_CORE C736 name=collection1 org.apache.solr.core.SolrCore@53539c88 url=http://127.0.0.1:50866/x_avr/bb/collection1 node=127.0.0.1:50866_x_avr%2Fbb C736_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:50866_x_avr%2Fbb, base_url=http://127.0.0.1:50866/x_avr/bb}
[junit4:junit4] 2> 1811453 T5523 C736 P50866 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1811457 T5508 C733 P50863 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4] 2> 1811460 T5541 C735 P50870 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1811467 T5541 C735 P50870 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5941403d lockFactory=org.apache.lucene.store.NativeFSLockFactory@e8473b4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1811469 T5541 C735 P50870 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1811471 T5541 C735 P50870 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> C732_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:50877_x_avr%2Fbb, base_url=http://127.0.0.1:50877/x_avr/bb}
[junit4:junit4] 2> 1811468 T5558 C732 P50877 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1811473 T5541 C735 P50870 oass.SolrIndexSearcher.<init> Opening Searcher@5375b359 main
[junit4:junit4] 2> 1811477 T5541 C735 P50870 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1811470 T5523 C736 P50866 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a0cdad5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@26516c43),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1811478 T5523 C736 P50866 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1811479 T5523 C736 P50866 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 1811480 T5552 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5375b359 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 1811482 T5558 C732 P50877 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2c5554a3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2b36df82),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1811483 T5558 C732 P50877 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1811485 T5558 C732 P50877 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 1811484 T5541 C735 P50870 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 24
[junit4:junit4] 2> 1811487 T5558 C732 P50877 oass.SolrIndexSearcher.<init> Opening Searcher@3071f3a main
[junit4:junit4] 2> 1811489 T5558 C732 P50877 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1811485 T5523 C736 P50866 oass.SolrIndexSearcher.<init> Opening Searcher@7f5b7ee4 main
[junit4:junit4] 2> 1811491 T5523 C736 P50866 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1811491 T5570 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3071f3a main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 1811493 T5558 C732 P50877 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 25
[junit4:junit4] 2> 1811494 T5534 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7f5b7ee4 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 1811496 T5523 C736 P50866 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 43
[junit4:junit4] 2> 1811497 T5540 C735 P50870 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 79
[junit4:junit4] 2> 1811501 T5474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1811509 T5511 C733 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 1811518 T5524 C736 P50866 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=2
[junit4:junit4] 2> 1811524 T5542 C735 P50870 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1811531 T5560 C732 P50877 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 1813541 T5492 C734 P50859 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439081010029395968)} 0 3
[junit4:junit4] 2> 1813567 T5542 C735 P50870 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={update.distrib=FROMLEADER&_version_=-1439081010038833152&update.from=http://127.0.0.1:50863/x_avr/bb/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439081010038833152)} 0 5
[junit4:junit4] 2> 1813570 T5524 C736 P50866 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={update.distrib=FROMLEADER&_version_=-1439081010038833152&update.from=http://127.0.0.1:50863/x_avr/bb/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439081010038833152)} 0 6
[junit4:junit4] 2> 1813572 T5560 C732 P50877 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={update.distrib=FROMLEADER&_version_=-1439081010038833152&update.from=http://127.0.0.1:50863/x_avr/bb/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439081010038833152)} 0 10
[junit4:junit4] 2> 1813574 T5511 C733 P50863 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439081010038833152)} 0 28
[junit4:junit4] 2> 1813586 T5487 C734 P50859 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={wt=javabin&version=2} {add=[0 (1439081010075533312)]} 0 5
[junit4:junit4] 2> 1813605 T5544 C735 P50870 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50863/x_avr/bb/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1439081010090213376)]} 0 2
[junit4:junit4] 2> 1813610 T5525 C736 P50866 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50863/x_avr/bb/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1439081010090213376)]} 0 2
[junit4:junit4] 2> 1813612 T5561 C732 P50877 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50863/x_avr/bb/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1439081010090213376)]} 0 4
[junit4:junit4] 2> 1813615 T5510 C733 P50863 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50870/x_avr/bb/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1439081010090213376)]} 0 19
[junit4:junit4] 2> 1813617 T5543 C735 P50870 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={wt=javabin&version=2} {add=[0]} 0 26
[junit4:junit4] 2> 1813626 T5488 C734 P50859 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={wt=javabin&version=2} {add=[1 (1439081010118524928)]} 0 4
[junit4:junit4] 2> 1813645 T5562 C732 P50877 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50863/x_avr/bb/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439081010129010688)]} 0 3
[junit4:junit4] 2> 1813652 T5526 C736 P50866 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50863/x_avr/bb/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439081010129010688)]} 0 3
[junit4:junit4] 2> 1813653 T5545 C735 P50870 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50863/x_avr/bb/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1439081010129010688)]} 0 3
[junit4:junit4] 2> 1813657 T5509 C733 P50863 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={wt=javabin&version=2} {add=[1 (1439081010129010688)]} 0 25
[junit4:junit4] 2> 1813667 T5489 C734 P50859 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={wt=javabin&version=2} {add=[2 (1439081010161516544)]} 0 4
[junit4:junit4] 2> 1813682 T5512 C733 P50863 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:50866/x_avr/bb/collection1/]
[junit4:junit4] 2> 1813683 T5512 C733 P50863 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:50866/x_avr/bb/collection1/ against:[http://127.0.0.1:50866/x_avr/bb/collection1/] result:true
[junit4:junit4] 2> 1813684 T5512 C733 P50863 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:50870/x_avr/bb/collection1/ against:[http://127.0.0.1:50866/x_avr/bb/collection1/] result:false
[junit4:junit4] 2> 1813684 T5512 C733 P50863 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:50877/x_avr/bb/collection1/ against:[http://127.0.0.1:50866/x_avr/bb/collection1/] result:false
[junit4:junit4] 2> 1813698 T5557 C732 P50877 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50863/x_avr/bb/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:50866/x_avr/bb/collection1/&wt=javabin&version=2} {add=[2 (1439081010183536640)]} 0 4
[junit4:junit4] 2> 1813705 T5541 C735 P50870 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50863/x_avr/bb/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:50866/x_avr/bb/collection1/&wt=javabin&version=2} {add=[2 (1439081010183536640)]} 0 4
[junit4:junit4] 2> 1813707 T5512 C733 P50863 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50866/x_avr/bb/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:50866/x_avr/bb/collection1/&wt=javabin&version=2} {add=[2 (1439081010183536640)]} 0 26
[junit4:junit4] 2> 1813710 T5527 C736 P50866 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={test.distrib.skip.servers=http://127.0.0.1:50866/x_avr/bb/collection1/&wt=javabin&version=2} {add=[2]} 0 37
[junit4:junit4] 2> 1813732 T5490 C734 P50859 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={wt=javabin&version=2} {add=[3 (1439081010216042496)]} 0 16
[junit4:junit4] 2> 1813742 T5507 C733 P50863 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:50866/x_avr/bb/collection1/]
[junit4:junit4] 2> 1813743 T5507 C733 P50863 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:50866/x_avr/bb/collection1/ against:[http://127.0.0.1:50866/x_avr/bb/collection1/] result:true
[junit4:junit4] 2> 1813744 T5507 C733 P50863 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:50870/x_avr/bb/collection1/ against:[http://127.0.0.1:50866/x_avr/bb/collection1/] result:false
[junit4:junit4] 2> 1813745 T5507 C733 P50863 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:50877/x_avr/bb/collection1/ against:[http://127.0.0.1:50866/x_avr/bb/collection1/] result:false
[junit4:junit4] 2> 1813754 T5542 C735 P50870 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50863/x_avr/bb/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:50866/x_avr/bb/collection1/&wt=javabin&version=2} {add=[3 (1439081010246451200)]} 0 3
[junit4:junit4] 2> 1813755 T5558 C732 P50877 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50863/x_avr/bb/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:50866/x_avr/bb/collection1/&wt=javabin&version=2} {add=[3 (1439081010246451200)]} 0 3
[junit4:junit4] 2> 1813757 T5507 C733 P50863 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50870/x_avr/bb/collection1/&update.distrib=TOLEADER&test.distrib.skip.servers=http://127.0.0.1:50866/x_avr/bb/collection1/&wt=javabin&version=2} {add=[3 (1439081010246451200)]} 0 15
[junit4:junit4] 2> 1813759 T5540 C735 P50870 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={test.distrib.skip.servers=http://127.0.0.1:50866/x_avr/bb/collection1/&test.distrib.skip.servers=http://127.0.0.1:50870/x_avr/bb/collection1/&wt=javabin&version=2} {add=[3]} 0 22
[junit4:junit4] 2> 1813764 T5491 C734 P50859 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1813779 T5491 C734 P50859 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b6ffa80 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2474b185),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b6ffa80 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2474b185),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1813781 T5491 C734 P50859 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1813787 T5491 C734 P50859 oass.SolrIndexSearcher.<init> Opening Searcher@79bb6355 main
[junit4:junit4] 2> 1813788 T5491 C734 P50859 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1813790 T5500 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@79bb6355 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 1813791 T5491 C734 P50859 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 27
[junit4:junit4] 2> 1813799 T5528 C736 P50866 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:50863/x_avr/bb/collection1/, StdNode: http://127.0.0.1:50866/x_avr/bb/collection1/, StdNode: http://127.0.0.1:50870/x_avr/bb/collection1/, StdNode: http://127.0.0.1:50877/x_avr/bb/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 1813805 T5508 C733 P50863 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1813813 T5523 C736 P50866 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1813816 T5559 C732 P50877 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1813826 T5544 C735 P50870 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1813845 T5559 C732 P50877 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2c5554a3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2b36df82),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2c5554a3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2b36df82),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1813852 T5559 C732 P50877 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1813858 T5508 C733 P50863 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4bcd36c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e431790),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4bcd36c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e431790),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1813859 T5508 C733 P50863 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1813861 T5559 C732 P50877 oass.SolrIndexSearcher.<init> Opening Searcher@1308d2a0 main
[junit4:junit4] 2> 1813862 T5559 C732 P50877 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1813864 T5570 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1308d2a0 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 1813865 T5559 C732 P50877 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 49
[junit4:junit4] 2> 1813872 T5508 C733 P50863 oass.SolrIndexSearcher.<init> Opening Searcher@516fecce main
[junit4:junit4] 2> 1813874 T5518 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@516fecce main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 1813875 T5508 C733 P50863 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1813875 T5508 C733 P50863 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 70
[junit4:junit4] 2> 1813879 T5523 C736 P50866 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a0cdad5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@26516c43),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a0cdad5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@26516c43),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1813880 T5523 C736 P50866 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1813883 T5523 C736 P50866 oass.SolrIndexSearcher.<init> Opening Searcher@7b33f285 main
[junit4:junit4] 2> 1813885 T5544 C735 P50870 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5941403d lockFactory=org.apache.lucene.store.NativeFSLockFactory@e8473b4),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5941403d lockFactory=org.apache.lucene.store.NativeFSLockFactory@e8473b4),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1813885 T5523 C736 P50866 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1813886 T5544 C735 P50870 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1813888 T5534 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@7b33f285 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C2)}
[junit4:junit4] 2> 1813889 T5544 C735 P50870 oass.SolrIndexSearcher.<init> Opening Searcher@1ecd6cbb main
[junit4:junit4] 2> 1813890 T5523 C736 P50866 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 77
[junit4:junit4] 2> 1813892 T5544 C735 P50870 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1813894 T5552 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1ecd6cbb main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 1813896 T5544 C735 P50870 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 70
[junit4:junit4] 2> 1813901 T5528 C736 P50866 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 103
[junit4:junit4] 2> 1813903 T5474 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 1813910 T5474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1813913 T5474 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 1813918 T5511 C733 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 1813926 T5524 C736 P50866 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=2 status=0 QTime=0
[junit4:junit4] 2> 1813936 T5543 C735 P50870 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 1813944 T5560 C732 P50877 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 1813946 T5474 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1813978 T5541 oasha.CollectionsHandler.handleSyncShardAction Syncing shard : shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2
[junit4:junit4] 2> 1813979 T5541 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1814030 T5509 oasha.CoreAdminHandler.handleRequestSyncAction I have been requested to sync up my shard
[junit4:junit4] 2> 1814031 T5509 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50863/x_avr/bb/collection1/
[junit4:junit4] 2> 1814031 T5509 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50863/x_avr/bb START replicas=[http://127.0.0.1:50866/x_avr/bb/collection1/, http://127.0.0.1:50870/x_avr/bb/collection1/, http://127.0.0.1:50877/x_avr/bb/collection1/] nUpdates=100
[junit4:junit4] 2> 1814042 T5542 C735 P50870 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1814043 T5526 C736 P50866 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 1814044 T5509 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50863/x_avr/bb Received 5 versions from 127.0.0.1:50870/x_avr/bb/collection1/
[junit4:junit4] 2> 1814045 T5509 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50863/x_avr/bb Our versions are newer. ourLowThreshold=1439081010038833152 otherHigh=1439081010183536640
[junit4:junit4] 2> 1814046 T5509 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50863/x_avr/bb Received 3 versions from 127.0.0.1:50866/x_avr/bb/collection1/
[junit4:junit4] 2> 1814046 T5509 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50863/x_avr/bb Our versions are newer. ourLowThreshold=1439081010038833152 otherHigh=1439081010129010688
[junit4:junit4] 2> 1814045 T5562 C732 P50877 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1814049 T5509 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50863/x_avr/bb Received 5 versions from 127.0.0.1:50877/x_avr/bb/collection1/
[junit4:junit4] 2> 1814049 T5509 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50863/x_avr/bb Our versions are newer. ourLowThreshold=1439081010038833152 otherHigh=1439081010183536640
[junit4:junit4] 2> 1814050 T5509 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50863/x_avr/bb DONE. sync succeeded
[junit4:junit4] 2> 1814050 T5509 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1814050 T5509 oasc.SyncStrategy.syncToMe http://127.0.0.1:50863/x_avr/bb/collection1/: try and ask http://127.0.0.1:50866/x_avr/bb/collection1/ to sync
[junit4:junit4] 2> 1814052 T5509 oasc.SyncStrategy.syncToMe http://127.0.0.1:50863/x_avr/bb/collection1/: try and ask http://127.0.0.1:50870/x_avr/bb/collection1/ to sync
[junit4:junit4] 2> 1814053 T5509 oasc.SyncStrategy.syncToMe http://127.0.0.1:50863/x_avr/bb/collection1/: try and ask http://127.0.0.1:50877/x_avr/bb/collection1/ to sync
[junit4:junit4] 2> 1814059 T5527 C736 P50866 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50866/x_avr/bb START replicas=[http://127.0.0.1:50863/x_avr/bb/collection1/] nUpdates=100
[junit4:junit4] 2> 1814062 T5540 C735 P50870 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50870/x_avr/bb START replicas=[http://127.0.0.1:50863/x_avr/bb/collection1/] nUpdates=100
[junit4:junit4] 2> 1814062 T5557 C732 P50877 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50877/x_avr/bb START replicas=[http://127.0.0.1:50863/x_avr/bb/collection1/] nUpdates=100
[junit4:junit4] 2> 1814067 T5512 C733 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1814071 T5512 C733 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1814077 T5527 C736 P50866 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50866/x_avr/bb Received 5 versions from 127.0.0.1:50863/x_avr/bb/collection1/
[junit4:junit4] 2> 1814078 T5540 C735 P50870 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50870/x_avr/bb Received 5 versions from 127.0.0.1:50863/x_avr/bb/collection1/
[junit4:junit4] 2> 1814080 T5540 C735 P50870 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50870/x_avr/bb Our versions are newer. ourLowThreshold=1439081010038833152 otherHigh=1439081010183536640
[junit4:junit4] 2> 1814081 T5540 C735 P50870 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50870/x_avr/bb DONE. sync succeeded
[junit4:junit4] 2> 1814079 T5527 C736 P50866 oasu.PeerSync.requestUpdates PeerSync: core=collection1 url=http://127.0.0.1:50866/x_avr/bb Requesting updates from 127.0.0.1:50863/x_avr/bb/collection1/n=2 versions=[1439081010246451200, 1439081010183536640]
[junit4:junit4] 2> 1814082 T5540 C735 P50870 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={sync=http://127.0.0.1:50863/x_avr/bb/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=20
[junit4:junit4] 2> 1814081 T5508 C733 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 1814085 T5509 oasc.SyncStrategy.syncToMe http://127.0.0.1:50863/x_avr/bb/collection1/: sync completed with http://127.0.0.1:50870/x_avr/bb/collection1/
[junit4:junit4] 2> 1814086 T5557 C732 P50877 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50877/x_avr/bb Received 5 versions from 127.0.0.1:50863/x_avr/bb/collection1/
[junit4:junit4] 2> 1814086 T5557 C732 P50877 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50877/x_avr/bb Our versions are newer. ourLowThreshold=1439081010038833152 otherHigh=1439081010183536640
[junit4:junit4] 2> 1814087 T5557 C732 P50877 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50877/x_avr/bb DONE. sync succeeded
[junit4:junit4] 2> 1814088 T5557 C732 P50877 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={sync=http://127.0.0.1:50863/x_avr/bb/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=26
[junit4:junit4] 2> 1814089 T5511 C733 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={distrib=false&getUpdates=1439081010246451200,1439081010183536640&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 1814090 T5509 oasc.SyncStrategy.syncToMe http://127.0.0.1:50863/x_avr/bb/collection1/: sync completed with http://127.0.0.1:50877/x_avr/bb/collection1/
[junit4:junit4] 2> 1814097 T5527 C736 P50866 oasup.LogUpdateProcessor.finish [collection1] {add=[2 (1439081010183536640), 3 (1439081010246451200)]} 0 6
[junit4:junit4] 2> 1814098 T5527 C736 P50866 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50866/x_avr/bb DONE. sync succeeded
[junit4:junit4] 2> 1814099 T5527 C736 P50866 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={sync=http://127.0.0.1:50863/x_avr/bb/collection1/&getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=40
[junit4:junit4] 2> 1814100 T5509 oasc.SyncStrategy.syncToMe http://127.0.0.1:50863/x_avr/bb/collection1/: sync completed with http://127.0.0.1:50866/x_avr/bb/collection1/
[junit4:junit4] 2> 1814102 T5509 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={shard=shard1&action=REQUESTSYNCSHARD&core=collection1&collection=collection1&wt=javabin&version=2} status=0 QTime=71
[junit4:junit4] 2> 1814103 T5541 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={shard=shard1&action=SYNCSHARD&collection=collection1&wt=javabin&version=2} status=0 QTime=125
[junit4:junit4] 2> 1814104 T5474 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 1814104 T5474 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 1814106 T5474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1814108 T5474 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> 1814112 T5487 C734 P50859 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1814113 T5487 C734 P50859 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 1814114 T5487 C734 P50859 oass.SolrIndexSearcher.<init> Opening Searcher@126f973a main
[junit4:junit4] 2> 1814115 T5487 C734 P50859 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1814116 T5500 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@126f973a main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 1814117 T5487 C734 P50859 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 5
[junit4:junit4] 2> 1814122 T5558 C732 P50877 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:50863/x_avr/bb/collection1/, StdNode: http://127.0.0.1:50866/x_avr/bb/collection1/, StdNode: http://127.0.0.1:50870/x_avr/bb/collection1/, StdNode: http://127.0.0.1:50877/x_avr/bb/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 1814133 T5510 C733 P50863 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1814138 T5510 C733 P50863 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 1814142 T5510 C733 P50863 oass.SolrIndexSearcher.<init> Opening Searcher@17e0f51 main
[junit4:junit4] 2> 1814142 T5510 C733 P50863 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1814143 T5523 C736 P50866 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1814145 T5518 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@17e0f51 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 1814146 T5559 C732 P50877 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1814146 T5559 C732 P50877 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 1814145 T5544 C735 P50870 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1814147 T5544 C735 P50870 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 1814148 T5510 C733 P50863 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 16
[junit4:junit4] 2> 1814149 T5544 C735 P50870 oass.SolrIndexSearcher.<init> Opening Searcher@2f6002c6 main
[junit4:junit4] 2> 1814150 T5544 C735 P50870 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1814155 T5523 C736 P50866 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a0cdad5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@26516c43),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@6a0cdad5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@26516c43),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1814156 T5523 C736 P50866 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1814157 T5559 C732 P50877 oass.SolrIndexSearcher.<init> Opening Searcher@3d627bff main
[junit4:junit4] 2> 1814158 T5559 C732 P50877 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1814159 T5552 oasc.JmxMonitoredMap.put WARN Failed to register info bean: searcher javax.management.InstanceAlreadyExistsException: solr/collection1:type=searcher,id=org.apache.solr.search.SolrIndexSearcher
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 2> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 2> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 2> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 2> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:305)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1844)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:135)
[junit4:junit4] 2> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1721)
[junit4:junit4] 2> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
[junit4:junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2>
[junit4:junit4] 2> 1814159 T5570 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3d627bff main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 1814161 T5559 C732 P50877 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 15
[junit4:junit4] 2> 1814161 T5552 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2f6002c6 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C4)}
[junit4:junit4] 2> 1814163 T5544 C735 P50870 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 18
[junit4:junit4] 2> 1814164 T5523 C736 P50866 oass.SolrIndexSearcher.<init> Opening Searcher@3ed009fc main
[junit4:junit4] 2> 1814165 T5523 C736 P50866 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1814167 T5534 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3ed009fc main{StandardDirectoryReader(segments_3:5:nrt _0(5.0):C2 _1(5.0):C2)}
[junit4:junit4] 2> 1814169 T5523 C736 P50866 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 26
[junit4:junit4] 2> 1814170 T5558 C732 P50877 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 49
[junit4:junit4] 2> 1814173 T5474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1814185 T5507 C733 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> 1814192 T5528 C736 P50866 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 1814197 T5545 C735 P50870 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 1814203 T5560 C732 P50877 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> 1816213 T5474 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> check const of shard1
[junit4:junit4] 2> client0
[junit4:junit4] 2> PROPS:core_node1:{"state":"active","core":"collection1","node_name":"127.0.0.1:50863_x_avr%2Fbb","base_url":"http://127.0.0.1:50863/x_avr/bb","leader":"true"}
[junit4:junit4] 2> 1816225 T5507 C733 P50863 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=0
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client1
[junit4:junit4] 2> PROPS:core_node2:{"state":"active","core":"collection1","node_name":"127.0.0.1:50866_x_avr%2Fbb","base_url":"http://127.0.0.1:50866/x_avr/bb"}
[junit4:junit4] 2> 1816230 T5528 C736 P50866 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client2
[junit4:junit4] 2> PROPS:core_node3:{"state":"active","core":"collection1","node_name":"127.0.0.1:50870_x_avr%2Fbb","base_url":"http://127.0.0.1:50870/x_avr/bb"}
[junit4:junit4] 2> 1816235 T5545 C735 P50870 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> client3
[junit4:junit4] 2> PROPS:core_node4:{"state":"active","core":"collection1","node_name":"127.0.0.1:50877_x_avr%2Fbb","base_url":"http://127.0.0.1:50877/x_avr/bb"}
[junit4:junit4] 2> 1816240 T5560 C732 P50877 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=4 status=0 QTime=1
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:4
[junit4:junit4] 2>
[junit4:junit4] 2> 1816245 T5524 C736 P50866 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/select params={q=*:*&wt=javabin&version=2} hits=4 status=0 QTime=0
[junit4:junit4] 2> 1816256 T5487 C734 P50859 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={wt=javabin&version=2} {add=[4 (1439081012874182656)]} 0 5
[junit4:junit4] 2> 1816262 T5512 C733 P50863 oasup.DistributedUpdateProcessor.setupRequest test.distrib.skip.servers was found and contains:[http://127.0.0.1:50877/x_avr/bb/collection1/]
[junit4:junit4] 2> 1816262 T5512 C733 P50863 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:50866/x_avr/bb/collection1/ against:[http://127.0.0.1:50877/x_avr/bb/collection1/] result:false
[junit4:junit4] 2> 1816263 T5512 C733 P50863 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:50870/x_avr/bb/collection1/ against:[http://127.0.0.1:50877/x_avr/bb/collection1/] result:false
[junit4:junit4] 2> 1816264 T5512 C733 P50863 oasup.DistributedUpdateProcessor.setupRequest check url:http://127.0.0.1:50877/x_avr/bb/collection1/ against:[http://127.0.0.1:50877/x_avr/bb/collection1/] result:true
[junit4:junit4] 2> 1816278 T5543 C735 P50870 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50863/x_avr/bb/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:50877/x_avr/bb/collection1/&wt=javabin&version=2} {add=[4 (1439081012888862720)]} 0 4
[junit4:junit4] 2> 1816278 T5525 C736 P50866 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={distrib.from=http://127.0.0.1:50863/x_avr/bb/collection1/&update.distrib=FROMLEADER&test.distrib.skip.servers=http://127.0.0.1:50877/x_avr/bb/collection1/&wt=javabin&version=2} {add=[4 (1439081012888862720)]} 0 5
[junit4:junit4] 2> 1816281 T5512 C733 P50863 oasup.LogUpdateProcessor.finish [collection1] webapp=/x_avr/bb path=/update params={test.distrib.skip.servers=http://127.0.0.1:50877/x_avr/bb/collection1/&wt=javabin&version=2} {add=[4 (1439081012888862720)]} 0 20
[junit4:junit4] 2> 1816282 T5474 oasc.ChaosMonkey.monkeyLog monkey: kill shard! 50863
[junit4:junit4] 2> 1816298 T5474 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=271335508
[junit4:junit4] 2> 1817374 T5497 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1817376 T5497 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "node_name":"127.0.0.1:50863_x_avr%2Fbb",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50863/x_avr/bb"}
[junit4:junit4] 2> 1817376 T5497 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
[junit4:junit4] 2> 1817388 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1817388 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1817389 T5532 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1817389 T5549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1817390 T5567 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1817389 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1819314 T5474 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 1819315 T5474 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 1819319 T5474 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@590d2c5f
[junit4:junit4] 2> 1819329 T5474 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=6,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=1,adds=1,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=5,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 1819330 T5474 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 1819330 T5474 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 1819331 T5474 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> C733_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:50863_x_avr%2Fbb, base_url=http://127.0.0.1:50863/x_avr/bb, leader=true}
[junit4:junit4] 2> 1819342 T5474 C733 P50863 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=3
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4bcd36c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e431790),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4bcd36c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e431790),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4bcd36c0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@7e431790),segFN=segments_3,generation=3}
[junit4:junit4] 2> 1819344 T5474 C733 P50863 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 1819351 T5474 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 1819353 T5474 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 1819354 T5474 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty1 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty1;done=false>>]
[junit4:junit4] 2> 1819356 T5474 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty1
[junit4:junit4] 2> 1819357 T5474 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty1/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty1/index;done=false>>]
[junit4:junit4] 2> 1819358 T5474 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.SyncSliceTest-1372414566268/jetty1/index
[junit4:junit4] 2> 1819366 T5567 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1819367 T5516 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1819367 T5516 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1819368 T5516 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 1819366 T5503 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1819366 T5549 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1819366 T5496 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1819369 T5474 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/x_avr/bb,null}
[junit4:junit4] 2> 1819383 T5567 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1819383 T5503 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1819386 T5549 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1819388 T5496 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1819388 T5532 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 1819397 T5532 oasc.ShardLeaderElectionContext.shouldIBeLeader Checking if I should try and be the leader.
[junit4:junit4] 2> 1819398 T5532 oasc.ShardLeaderElectionContext.shouldIBeLeader My last published State was Active, it's okay to be the leader.
[junit4:junit4] 2> 1819398 T5532 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1819399 T5532 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50866/x_avr/bb/collection1/
[junit4:junit4] 2> 1819399 T5532 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50866/x_avr/bb START replicas=[http://127.0.0.1:50863/x_avr/bb/collection1/, http://127.0.0.1:50870/x_avr/bb/collection1/, http://127.0.0.1:50877/x_avr/bb/collection1/] nUpdates=100
[junit4:junit4] 2> 1819410 T5541 C735 P50870 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1819412 T5532 oasu.PeerSync.handleResponse WARN PeerSync: core=collection1 url=http://127.0.0.1:50866/x_avr/bb couldn't connect to http://127.0.0.1:50863/x_avr/bb/collection1/, counting as success
[junit4:junit4] 2> 1819413 T5532 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50866/x_avr/bb Received 6 versions from 127.0.0.1:50870/x_avr/bb/collection1/
[junit4:junit4] 2> 1819414 T5532 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50866/x_avr/bb Our versions are newer. ourLowThreshold=1439081010090213376 otherHigh=1439081010246451200
[junit4:junit4] 2> 1819410 T5557 C732 P50877 oasc.SolrCore.execute [collection1] webapp=/x_avr/bb path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 1819417 T5532 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50866/x_avr/bb Received 5 versions from 127.0.0.1:50877/x_avr/bb/collection1/
[junit4:junit4] 2> 1819418 T5532 oasu.PeerSync.handleVersions PeerSync: core=collection1 url=http://127.0.0.1:50866/x_avr/bb Our versions are newer. ourLowThreshold=1439081010090213376 otherHigh=1439081010183536640
[junit4:junit4] 2> 1819419 T5532 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:50866/x_avr/bb DONE. sync succeeded
[junit4:junit4] 2> 1819419 T5532 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 1819420 T5532 oasc.SyncStrategy.syncToMe http://127.0.0.1:50866/x_avr/bb/collection1/: try and ask http://127.0.0.1:50863/x_avr/bb/collection1/ to sync
[junit4:junit4] 2> 1819421 T5532 oasc.SyncStrategy.syncToMe http://127.0.0.1:50866/x_avr/bb/collection1/: try and ask http://127.0.0.1:50870/x_avr/bb/collection1/ to sync
[junit4:junit4] 2> 1819423 T5532 oasc.SyncStrategy.syncToMe http://127.0.0.1:5
[...truncated too long message...]
ons/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89942561015595022-core_node1-n_0000000004 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89942561015595020-core_node4-n_0000000003 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89942561015595018-core_node3-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (1)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:50870_x_avr%2Fbb",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:50870/x_avr/bb"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89942561015595011-core_node1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "node_name":"127.0.0.1:50859_x_avr%2Fbb",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:50859/x_avr/bb"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (4)
[junit4:junit4] 1> /solr/overseer_elect/election/89942561015595011-127.0.0.1:50859_x_avr%2Fbb-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89942561015595022-127.0.0.1:50863_x_avr%2Fbb-n_0000000005 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89942561015595020-127.0.0.1:50877_x_avr%2Fbb-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89942561015595018-127.0.0.1:50870_x_avr%2Fbb-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89942561015595011-127.0.0.1:50859_x_avr%2Fbb-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SyncSliceTest -Dtests.method=testDistribSearch -Dtests.seed=60679E00F3D4D777 -Dtests.slow=true -Dtests.locale=hr -Dtests.timezone=Asia/Ujung_Pandang -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 119s | SyncSliceTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard1 is not consistent. Got 305 from http://127.0.0.1:50863/x_avr/bb/collection1lastClient and got 248 from http://127.0.0.1:50870/x_avr/bb/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([60679E00F3D4D777:E1811018848BB74B]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1045)
[junit4:junit4] > at org.apache.solr.cloud.SyncSliceTest.doTest(SyncSliceTest.java:237)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2> 1899787 T5474 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 118625 T5473 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {range_facet_l=PostingsFormat(name=MockSep), text=PostingsFormat(name=Asserting), _version_=PostingsFormat(name=Memory doPackFST= false), rnd_b=Pulsing41(freqCutoff=16 minBlockSize=35 maxBlockSize=149), intDefault=PostingsFormat(name=Memory doPackFST= false), timestamp=PostingsFormat(name=MockSep), id=Pulsing41(freqCutoff=16 minBlockSize=35 maxBlockSize=149), a_t=PostingsFormat(name=MockSep), range_facet_sl=Pulsing41(freqCutoff=16 minBlockSize=35 maxBlockSize=149), range_facet_si=PostingsFormat(name=MockSep), other_tl1=PostingsFormat(name=Memory doPackFST= false), multiDefault=PostingsFormat(name=MockSep), a_si=PostingsFormat(name=Memory doPackFST= false)}, docValues:{timestamp=DocValuesFormat(name=Lucene42)}, sim=DefaultSimilarity, locale=hr, timezone=Asia/Ujung_Pandang
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=2,threads=2,free=282285712,total=535822336
[junit4:junit4] 2> NOTE: All tests run in this JVM: [HdfsBasicDistributedZk2Test, MBeansHandlerTest, TestRecoveryHdfs, PrimitiveFieldTypeTest, QueryResultKeyTest, RecoveryZkTest, CurrencyFieldOpenExchangeTest, ShardRoutingCustomTest, TestSchemaSimilarityResource, AssignTest, TestIndexSearcher, DistributedTermsComponentTest, TestSystemIdResolver, TestZkChroot, PrimUtilsTest, HdfsSyncSliceTest, DirectUpdateHandlerTest, ClusterStateTest, SampleTest, LukeRequestHandlerTest, TestFieldResource, TestArbitraryIndexDir, TestFieldCollectionResource, BadIndexSchemaTest, TestSuggestSpellingConverter, DirectUpdateHandlerOptimizeTest, TestDFRSimilarityFactory, TestJoin, TestCSVResponseWriter, SpatialFilterTest, TestSerializedLuceneMatchVersion, SpellPossibilityIteratorTest, TestBadConfig, SchemaVersionSpecificBehaviorTest, StandardRequestHandlerTest, HighlighterConfigTest, TestRTGBase, TestLazyCores, SliceStateTest, TermsComponentTest, HdfsBasicDistributedZkTest, TestFiltering, LeaderElectionTest, TestStressReorder, HdfsChaosMonkeySafeLeaderTest, SpellCheckComponentTest, TestSolrXMLSerializer, TestBinaryResponseWriter, UpdateRequestProcessorFactoryTest, TestPhraseSuggestions, TestSchemaNameResource, SliceStateUpdateTest, TestLMDirichletSimilarityFactory, TestManagedSchema, TestLuceneMatchVersion, SolrIndexSplitterTest, XmlUpdateRequestHandlerTest, TestSchemaResource, TestReload, TestFastLRUCache, TestPseudoReturnFields, WordBreakSolrSpellCheckerTest, JSONWriterTest, SpellingQueryConverterTest, TestCopyFieldCollectionResource, CoreContainerCoreInitFailuresTest, PolyFieldTest, TestTrie, UniqFieldsUpdateProcessorFactoryTest, TestOmitPositions, SystemInfoHandlerTest, CircularListTest, IndexReaderFactoryTest, CollectionsAPIDistributedZkTest, TestDefaultSearchFieldResource, TestFoldingMultitermQuery, TestSurroundQueryParser, DOMUtilTest, TestSweetSpotSimilarityFactory, UUIDFieldTest, DocValuesMultiTest, TestAnalyzedSuggestions, TestPostingsSolrHighlighter, FastVectorHighlighterTest, TestSolrXml, TestSolr4Spatial, TestDynamicFieldCollectionResource, SuggesterFSTTest, TestPropInject, TestIndexingPerformance, TermVectorComponentTest, DirectSolrConnectionTest, DirectSolrSpellCheckerTest, ZkNodePropsTest, TestMergePolicyConfig, TestMaxScoreQueryParser, OpenCloseCoreStressTest, ShowFileRequestHandlerTest, MoreLikeThisHandlerTest, HdfsDirectoryTest, LoggingHandlerTest, TestPropInjectDefaults, TestQueryUtils, BlockDirectoryTest, TestSolrIndexConfig, HighlighterTest, JsonLoaderTest, HdfsLockFactoryTest, TestExtendedDismaxParser, SolrCoreTest, LegacyHTMLStripCharFilterTest, LeaderElectionIntegrationTest, TestStressVersions, OverseerCollectionProcessorTest, TestIBSimilarityFactory, PathHierarchyTokenizerFactoryTest, TestCoreContainer, TestElisionMultitermQuery, TestSolrQueryParser, TestUtils, DocumentAnalysisRequestHandlerTest, BinaryUpdateRequestHandlerTest, TestStressRecovery, TestDistributedGrouping, TestComponentsName, TestSolrJ, OverseerTest, SuggesterTest, TestPerFieldSimilarity, TestSolrCoreProperties, TestQueryTypes, ZkControllerTest, SortByFunctionTest, ChaosMonkeyNothingIsSafeTest, AutoCommitTest, BlockCacheTest, BasicZkTest, DistributedSpellCheckComponentTest, TestPartialUpdateDeduplication, SynonymTokenizerTest, AliasIntegrationTest, TestCSVLoader, PluginInfoTest, TestSolrQueryParserResource, CacheHeaderTest, TestFunctionQuery, TestWriterPerf, SOLR749Test, ExternalFileFieldSortTest, TestRemoteStreaming, FieldAnalysisRequestHandlerTest, SyncSliceTest]
[junit4:junit4] Completed in 118.99s, 1 test, 1 failure <<< FAILURES!
[...truncated 496 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:190: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:443: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1245: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 1 failure, 25 ignored (12 assumptions)
Total time: 97 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 605 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/605/
Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseSerialGC
2 tests failed.
REGRESSION: org.apache.solr.cloud.AliasIntegrationTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:49224 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:49224 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([D6C4F79DCF3F97A1:57227985B860F79D]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.AliasIntegrationTest.deleteAlias(AliasIntegrationTest.java:253)
at org.apache.solr.cloud.AliasIntegrationTest.doTest(AliasIntegrationTest.java:226)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
REGRESSION: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:49498/rovu returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:49498/rovu returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([D6C4F79DCF3F97A1:57227985B860F79D]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:724)
Build Log:
[...truncated 9138 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.AliasIntegrationTest
[junit4:junit4] 2> 162500 T318 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 162514 T318 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-AliasIntegrationTest-1372438089621
[junit4:junit4] 2> 162516 T318 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 162518 T319 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 162531 T318 oasc.ZkTestServer.run start zk server on port:49217
[junit4:junit4] 2> 162540 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 162547 T325 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@44013712 name:ZooKeeperConnection Watcher:127.0.0.1:49217 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 162548 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 162548 T318 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 162594 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 162598 T327 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@43e304a3 name:ZooKeeperConnection Watcher:127.0.0.1:49217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 162599 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 162620 T318 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 162633 T318 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 162642 T318 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 162650 T318 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 162659 T318 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 162660 T318 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 162678 T318 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 162680 T318 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 162691 T318 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 162692 T318 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 162701 T318 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 162703 T318 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 162711 T318 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 162712 T318 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 162721 T318 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 162722 T318 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 162749 T318 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 162750 T318 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 162759 T318 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 162760 T318 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 162768 T318 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 162770 T318 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 163296 T318 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 163336 T318 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49220
[junit4:junit4] 2> 163337 T318 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 163338 T318 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 163338 T318 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886
[junit4:junit4] 2> 163339 T318 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/solr.xml
[junit4:junit4] 2> 163339 T318 oasc.CoreContainer.<init> New CoreContainer 859847647
[junit4:junit4] 2> 163340 T318 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/'
[junit4:junit4] 2> 163341 T318 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/'
[junit4:junit4] 2> 163583 T318 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 163583 T318 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 163584 T318 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 163585 T318 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 163585 T318 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 163586 T318 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 163586 T318 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 163587 T318 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 163587 T318 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 163588 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 163614 T318 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 163615 T318 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49217/solr
[junit4:junit4] 2> 163637 T318 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 163639 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 163645 T338 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2576561b name:ZooKeeperConnection Watcher:127.0.0.1:49217 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 163646 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 163653 T320 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13f8baf4fa30002, likely client has closed socket
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2>
[junit4:junit4] 2> 163659 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 163692 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 163698 T340 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@76a67dab name:ZooKeeperConnection Watcher:127.0.0.1:49217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 163699 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 163720 T318 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 163755 T318 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 163790 T318 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 163796 T318 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49220_
[junit4:junit4] 2> 163799 T318 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49220_
[junit4:junit4] 2> 163817 T318 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 163851 T318 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 163860 T318 oasc.Overseer.start Overseer (id=89944102642319363-127.0.0.1:49220_-n_0000000000) starting
[junit4:junit4] 2> 163895 T318 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 163939 T342 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 163951 T318 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 164008 T318 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 164030 T318 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 164088 T341 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 164121 T343 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 164122 T343 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 164125 T343 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 165621 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 165637 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49220_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49220"}
[junit4:junit4] 2> 165638 T341 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 165657 T341 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 165671 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 166143 T343 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 166144 T343 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection1
[junit4:junit4] 2> 166144 T343 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 166146 T343 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 166147 T343 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 166151 T343 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection1/'
[junit4:junit4] 2> 166152 T343 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 166153 T343 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection1/lib/README' to classloader
[junit4:junit4] 2> 166302 T343 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 166423 T343 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 166428 T343 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 166458 T343 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 168266 T343 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 168283 T343 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 168298 T343 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 168368 T343 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 168377 T343 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 168386 T343 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 168391 T343 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 168391 T343 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 168392 T343 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 168395 T343 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 168396 T343 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 168396 T343 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 168397 T343 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection1/, dataDir=./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/control/data/
[junit4:junit4] 2> 168397 T343 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44385e76
[junit4:junit4] 2> 168404 T343 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/control/data
[junit4:junit4] 2> 168404 T343 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/control/data/index/
[junit4:junit4] 2> 168406 T343 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.AliasIntegrationTest-1372438089620/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 168406 T343 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/control/data/index
[junit4:junit4] 2> 168412 T343 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67c860e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@69eb2cdb),segFN=segments_1,generation=1}
[junit4:junit4] 2> 168412 T343 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 168491 T343 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 168491 T343 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 168492 T343 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 168493 T343 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 168494 T343 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 168495 T343 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 168495 T343 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 168496 T343 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 168497 T343 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 168512 T343 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 168520 T343 oass.SolrIndexSearcher.<init> Opening Searcher@1545dd25 main
[junit4:junit4] 2> 168522 T343 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 168523 T343 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 168541 T344 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1545dd25 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 168543 T343 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 168543 T343 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49220 collection:control_collection shard:shard1
[junit4:junit4] 2> 168558 T343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=10000&maxConnectionsPerHost=20&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 168586 T343 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 168624 T343 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 168634 T343 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 168634 T343 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 168640 T343 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49220/collection1/
[junit4:junit4] 2> 168641 T343 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 168642 T343 oasc.SyncStrategy.syncToMe http://127.0.0.1:49220/collection1/ has no replicas
[junit4:junit4] 2> 168642 T343 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49220/collection1/ shard1
[junit4:junit4] 2> 168643 T343 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 168731 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 168769 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 168786 T343 oasc.ZkController.register We are http://127.0.0.1:49220/collection1/ and leader is http://127.0.0.1:49220/collection1/
[junit4:junit4] 2> 168787 T343 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49220
[junit4:junit4] 2> 168787 T343 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 168788 T343 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 168788 T343 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 168812 T343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 168819 T318 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 168819 T318 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 168826 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 168878 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 168896 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 168901 T347 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10ea355b name:ZooKeeperConnection Watcher:127.0.0.1:49217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 168902 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 168916 T318 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 168935 T318 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 169424 T318 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 169430 T318 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49224
[junit4:junit4] 2> 169432 T318 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 169433 T318 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 169433 T318 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041
[junit4:junit4] 2> 169434 T318 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/solr.xml
[junit4:junit4] 2> 169434 T318 oasc.CoreContainer.<init> New CoreContainer 733552213
[junit4:junit4] 2> 169435 T318 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/'
[junit4:junit4] 2> 169436 T318 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/'
[junit4:junit4] 2> 169654 T318 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 169656 T318 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 169657 T318 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 169658 T318 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 169658 T318 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 169659 T318 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 169660 T318 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 169660 T318 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 169661 T318 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 169662 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 169693 T318 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 169694 T318 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49217/solr
[junit4:junit4] 2> 169695 T318 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 169698 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 169704 T358 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cc94643 name:ZooKeeperConnection Watcher:127.0.0.1:49217 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 169705 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 169715 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 169744 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 169748 T360 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b071630 name:ZooKeeperConnection Watcher:127.0.0.1:49217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 169749 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 169766 T318 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 170296 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 170299 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49220_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49220"}
[junit4:junit4] 2> 170316 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 170323 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 170324 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 170781 T318 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49224_
[junit4:junit4] 2> 170783 T318 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49224_
[junit4:junit4] 2> 170791 T347 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 170791 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 170795 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 170809 T360 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 170809 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 170812 T340 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 170832 T361 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 170833 T361 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 170836 T361 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 171841 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 171843 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49224_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49224"}
[junit4:junit4] 2> 171844 T341 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 1
[junit4:junit4] 2> 171844 T341 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 171860 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 171872 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 171875 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 172838 T361 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 172839 T361 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection1
[junit4:junit4] 2> 172839 T361 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 172841 T361 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 172842 T361 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 172846 T361 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection1/'
[junit4:junit4] 2> 172847 T361 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 172848 T361 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection1/lib/README' to classloader
[junit4:junit4] 2> 172941 T361 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 173040 T361 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 173045 T361 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 173063 T361 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 174065 T361 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 174093 T361 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 174098 T361 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 174151 T361 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 174184 T361 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 174193 T361 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 174197 T361 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 174198 T361 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 174198 T361 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 174201 T361 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 174202 T361 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 174202 T361 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 174203 T361 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection1/, dataDir=./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty1/
[junit4:junit4] 2> 174203 T361 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44385e76
[junit4:junit4] 2> 174205 T361 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty1
[junit4:junit4] 2> 174231 T361 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty1/index/
[junit4:junit4] 2> 174231 T361 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 174232 T361 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty1/index
[junit4:junit4] 2> 174237 T361 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1672f0a6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18bdefd7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 174238 T361 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 174247 T361 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 174248 T361 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 174249 T361 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 174250 T361 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 174258 T361 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 174259 T361 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 174260 T361 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 174261 T361 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 174262 T361 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 174277 T361 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 174298 T361 oass.SolrIndexSearcher.<init> Opening Searcher@51ea6fe1 main
[junit4:junit4] 2> 174300 T361 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 174301 T361 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 174308 T362 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@51ea6fe1 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 174324 T361 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 174325 T361 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49224 collection:collection1 shard:shard1
[junit4:junit4] 2> 174356 T361 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 174396 T361 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 174413 T361 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 174413 T361 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 174414 T361 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49224/collection1/
[junit4:junit4] 2> 174415 T361 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 174415 T361 oasc.SyncStrategy.syncToMe http://127.0.0.1:49224/collection1/ has no replicas
[junit4:junit4] 2> 174415 T361 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49224/collection1/ shard1
[junit4:junit4] 2> 174416 T361 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 174925 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 174958 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 174961 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 174962 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 175002 T361 oasc.ZkController.register We are http://127.0.0.1:49224/collection1/ and leader is http://127.0.0.1:49224/collection1/
[junit4:junit4] 2> 175003 T361 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49224
[junit4:junit4] 2> 175004 T361 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 175004 T361 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 175004 T361 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 175010 T361 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 175014 T318 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 175014 T318 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 175015 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 175535 T318 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 175542 T318 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49227
[junit4:junit4] 2> 175543 T318 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 175544 T318 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 175544 T318 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty2-1372438102150
[junit4:junit4] 2> 175545 T318 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty2-1372438102150/solr.xml
[junit4:junit4] 2> 175546 T318 oasc.CoreContainer.<init> New CoreContainer 380687573
[junit4:junit4] 2> 175546 T318 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty2-1372438102150/'
[junit4:junit4] 2> 175547 T318 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty2-1372438102150/'
[junit4:junit4] 2> 175756 T318 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 175757 T318 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 175757 T318 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 175758 T318 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 175758 T318 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 175759 T318 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 175759 T318 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 175760 T318 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 175760 T318 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 175761 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 175783 T318 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 175784 T318 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49217/solr
[junit4:junit4] 2> 175785 T318 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 175791 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 175797 T374 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3dd764fe name:ZooKeeperConnection Watcher:127.0.0.1:49217 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 175798 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 175806 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 175832 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 175838 T376 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@70d7c55c name:ZooKeeperConnection Watcher:127.0.0.1:49217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 175838 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 175859 T318 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 176502 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 176506 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49224_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49224"}
[junit4:junit4] 2> 176528 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 176529 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 176537 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 176542 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 176871 T318 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49227_
[junit4:junit4] 2> 176875 T318 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49227_
[junit4:junit4] 2> 176888 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 176895 T340 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 176897 T360 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 176898 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 176904 T347 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 176904 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 176915 T376 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 176916 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 176934 T377 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 176934 T377 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 176951 T377 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 178064 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 178067 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49227_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49227"}
[junit4:junit4] 2> 178067 T341 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 178076 T341 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 178090 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 178091 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 178094 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 178102 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 178954 T377 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 178954 T377 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty2-1372438102150/collection1
[junit4:junit4] 2> 178954 T377 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 178956 T377 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 178956 T377 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 178962 T377 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty2-1372438102150/collection1/'
[junit4:junit4] 2> 178969 T377 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.AliasIntegrationTest-jetty2-1372438102150/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 178969 T377 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.AliasIntegrationTest-jetty2-1372438102150/collection1/lib/README' to classloader
[junit4:junit4] 2> 179043 T377 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 179136 T377 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 179140 T377 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 179157 T377 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 180158 T377 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 180182 T377 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 180188 T377 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 180369 T377 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 180388 T377 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 180397 T377 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 180400 T377 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 180401 T377 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 180402 T377 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 180405 T377 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 180406 T377 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 180406 T377 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 180407 T377 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty2-1372438102150/collection1/, dataDir=./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty2/
[junit4:junit4] 2> 180407 T377 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44385e76
[junit4:junit4] 2> 180409 T377 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty2
[junit4:junit4] 2> 180409 T377 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty2/index/
[junit4:junit4] 2> 180409 T377 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 180413 T377 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty2/index
[junit4:junit4] 2> 180424 T377 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4436f972 lockFactory=org.apache.lucene.store.NativeFSLockFactory@63b276e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 180424 T377 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 180430 T377 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 180430 T377 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 180432 T377 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 180433 T377 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 180435 T377 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 180435 T377 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 180436 T377 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 180437 T377 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 180438 T377 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 180462 T377 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 180470 T377 oass.SolrIndexSearcher.<init> Opening Searcher@f69f10 main
[junit4:junit4] 2> 180472 T377 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 180472 T377 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 180487 T378 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@f69f10 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 180492 T377 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 180492 T377 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49227 collection:collection1 shard:shard1
[junit4:junit4] 2> 180504 T377 oasc.ZkController.register We are http://127.0.0.1:49227/collection1/ and leader is http://127.0.0.1:49224/collection1/
[junit4:junit4] 2> 180505 T377 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49227
[junit4:junit4] 2> 180505 T377 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 180506 T377 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C213 name=collection1 org.apache.solr.core.SolrCore@3076f54c url=http://127.0.0.1:49227/collection1 node=127.0.0.1:49227_ C213_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:49227_, base_url=http://127.0.0.1:49227}
[junit4:junit4] 2> 180525 T379 C213 P49227 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 180526 T379 C213 P49227 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 180527 T379 C213 P49227 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 180527 T379 C213 P49227 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 180548 T379 C213 P49227 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 180568 T377 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 180636 T352 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 180641 T318 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 180642 T318 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 180642 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 181106 T318 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 181117 T318 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49231
[junit4:junit4] 2> 181118 T318 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 181119 T318 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 181119 T318 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty3-1372438107764
[junit4:junit4] 2> 181120 T318 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty3-1372438107764/solr.xml
[junit4:junit4] 2> 181120 T318 oasc.CoreContainer.<init> New CoreContainer 1805127664
[junit4:junit4] 2> 181121 T318 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty3-1372438107764/'
[junit4:junit4] 2> 181122 T318 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty3-1372438107764/'
[junit4:junit4] 2> 181156 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 181158 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49227_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49227"}
[junit4:junit4] 2> 181193 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 181193 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 181205 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 181209 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 181350 T318 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 181351 T318 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 181352 T318 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 181352 T318 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 181353 T318 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 181354 T318 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 181354 T318 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 181355 T318 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 181355 T318 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 181357 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 181380 T318 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 181381 T318 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49217/solr
[junit4:junit4] 2> 181381 T318 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 181385 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 181389 T391 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3072a301 name:ZooKeeperConnection Watcher:127.0.0.1:49217 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 181390 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 181395 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 181421 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 181426 T393 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@20be8aed name:ZooKeeperConnection Watcher:127.0.0.1:49217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 181426 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 181439 T318 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 181638 T352 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node2, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 181639 T352 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node2&state=recovering&nodeName=127.0.0.1:49227_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1009
[junit4:junit4] 2> 182450 T318 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49231_
[junit4:junit4] 2> 182453 T318 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49231_
[junit4:junit4] 2> 182460 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 182471 T360 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 182472 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 182478 T347 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 182478 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 182488 T376 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 182488 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 182497 T340 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 182501 T393 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 182510 T394 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 182510 T394 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 182513 T394 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 182720 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 182724 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49231_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49231"}
[junit4:junit4] 2> 182725 T341 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 182727 T341 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 182750 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 182751 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 182756 T393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 182756 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 182757 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 183515 T394 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 183516 T394 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty3-1372438107764/collection1
[junit4:junit4] 2> 183516 T394 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 183519 T394 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 183519 T394 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 183532 T394 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty3-1372438107764/collection1/'
[junit4:junit4] 2> 183534 T394 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.AliasIntegrationTest-jetty3-1372438107764/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 183534 T394 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.AliasIntegrationTest-jetty3-1372438107764/collection1/lib/README' to classloader
[junit4:junit4] 2> 183607 T394 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> ASYNC NEW_CORE C214 name=collection1 org.apache.solr.core.SolrCore@3076f54c url=http://127.0.0.1:49227/collection1 node=127.0.0.1:49227_ C214_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:49227_, base_url=http://127.0.0.1:49227}
[junit4:junit4] 2> 183642 T379 C214 P49227 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49224/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 183666 T379 C214 P49227 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&connTimeout=30000&socketTimeout=30000&retry=false
[junit4:junit4] 2> 183709 T379 C214 P49227 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49227 START replicas=[http://127.0.0.1:49224/collection1/] nUpdates=100
[junit4:junit4] 2> 183714 T379 C214 P49227 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 183728 T379 C214 P49227 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 183730 T379 C214 P49227 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 183735 T379 C214 P49227 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 183735 T379 C214 P49227 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 183736 T379 C214 P49227 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49224/collection1/. core=collection1
[junit4:junit4] 2> 183736 T379 C214 P49227 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C215 name=collection1 org.apache.solr.core.SolrCore@623ff96d url=http://127.0.0.1:49224/collection1 node=127.0.0.1:49224_ C215_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49224_, base_url=http://127.0.0.1:49224, leader=true}
[junit4:junit4] 2> 183785 T355 C215 P49224 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 183822 T394 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 183840 T394 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 183873 T394 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 184269 T356 C215 P49224 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 184329 T356 C215 P49224 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1672f0a6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18bdefd7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 184331 T356 C215 P49224 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 184331 T356 C215 P49224 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 184332 T356 C215 P49224 oass.SolrIndexSearcher.<init> Opening Searcher@3295cb9c realtime
[junit4:junit4] 2> 184333 T356 C215 P49224 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 184333 T356 C215 P49224 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 533
[junit4:junit4] 2> ASYNC NEW_CORE C216 name=collection1 org.apache.solr.core.SolrCore@3076f54c url=http://127.0.0.1:49227/collection1 node=127.0.0.1:49227_ C216_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:49227_, base_url=http://127.0.0.1:49227}
[junit4:junit4] 2> 184335 T379 C216 P49227 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 184336 T379 C216 P49227 oascsi.HttpClientUtil.createClient Creating new http client, config:connTimeout=5000&socketTimeout=20000&allowCompression=false&maxConnections=10000&maxConnectionsPerHost=10000
[junit4:junit4] 2> ASYNC NEW_CORE C217 name=collection1 org.apache.solr.core.SolrCore@3076f54c url=http://127.0.0.1:49227/collection1 node=127.0.0.1:49227_ C217_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:49227_, base_url=http://127.0.0.1:49227}
[junit4:junit4] 2> 184441 T379 C217 P49227 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> ASYNC NEW_CORE C218 name=collection1 org.apache.solr.core.SolrCore@623ff96d url=http://127.0.0.1:49224/collection1 node=127.0.0.1:49224_ C218_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49224_, base_url=http://127.0.0.1:49224, leader=true}
[junit4:junit4] 2> 184453 T354 C218 P49224 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 184454 T354 C218 P49224 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 184459 T379 C217 P49227 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 184460 T379 C217 P49227 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 184460 T379 C217 P49227 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 184460 T379 C217 P49227 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 184473 T379 C217 P49227 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 185357 T394 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 185380 T394 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 185386 T394 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 185440 T394 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 185449 T394 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 185484 T394 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 185489 T394 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 185491 T394 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 185491 T394 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 185495 T394 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 185496 T394 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 185496 T394 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 185497 T394 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty3-1372438107764/collection1/, dataDir=./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty3/
[junit4:junit4] 2> 185497 T394 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44385e76
[junit4:junit4] 2> 185498 T394 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty3
[junit4:junit4] 2> 185499 T394 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty3/index/
[junit4:junit4] 2> 185499 T394 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 185500 T394 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty3/index
[junit4:junit4] 2> 185506 T394 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@447088aa lockFactory=org.apache.lucene.store.NativeFSLockFactory@4472abe8),segFN=segments_1,generation=1}
[junit4:junit4] 2> 185506 T394 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 185511 T394 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 185512 T394 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 185513 T394 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 185514 T394 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 185516 T394 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 185516 T394 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 185523 T394 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 185524 T394 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 185525 T394 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 185539 T394 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 185556 T394 oass.SolrIndexSearcher.<init> Opening Searcher@417e412c main
[junit4:junit4] 2> 185558 T394 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 185558 T394 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 185566 T396 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@417e412c main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 185581 T394 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 185582 T394 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49231 collection:collection1 shard:shard1
[junit4:junit4] 2> 185594 T394 oasc.ZkController.register We are http://127.0.0.1:49231/collection1/ and leader is http://127.0.0.1:49224/collection1/
[junit4:junit4] 2> 185595 T394 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49231
[junit4:junit4] 2> 185597 T394 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 185598 T394 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C219 name=collection1 org.apache.solr.core.SolrCore@33f5c786 url=http://127.0.0.1:49231/collection1 node=127.0.0.1:49231_ C219_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:49231_, base_url=http://127.0.0.1:49231}
[junit4:junit4] 2> 185598 T397 C219 P49231 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 185600 T397 C219 P49231 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 185600 T397 C219 P49231 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 185600 T397 C219 P49231 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 185631 T394 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 185638 T397 C219 P49231 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 185662 T318 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 185666 T318 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 185667 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 185692 T356 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 185839 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 185842 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49227_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49227"}
[junit4:junit4] 2> 185854 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49231_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49231"}
[junit4:junit4] 2> 185885 T393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 185885 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 185885 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 185885 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 185896 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 186339 T318 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 186353 T318 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49238
[junit4:junit4] 2> 186355 T318 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 186356 T318 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 186356 T318 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty4-1372438112830
[junit4:junit4] 2> 186358 T318 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty4-1372438112830/solr.xml
[junit4:junit4] 2> 186358 T318 oasc.CoreContainer.<init> New CoreContainer 1511848054
[junit4:junit4] 2> 186359 T318 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty4-1372438112830/'
[junit4:junit4] 2> 186359 T318 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty4-1372438112830/'
[junit4:junit4] 2> 186553 T318 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 186554 T318 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 186555 T318 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 186555 T318 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 186556 T318 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 186556 T318 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 186557 T318 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 186557 T318 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 186558 T318 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 186558 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 186579 T318 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 186580 T318 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:49217/solr
[junit4:junit4] 2> 186581 T318 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 186583 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 186590 T409 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1066d85f name:ZooKeeperConnection Watcher:127.0.0.1:49217 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 186590 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 186597 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 186621 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 186626 T411 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a897294 name:ZooKeeperConnection Watcher:127.0.0.1:49217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 186626 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 186642 T318 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 186696 T356 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 186710 T356 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node3&state=recovering&nodeName=127.0.0.1:49231_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1018
[junit4:junit4] 2> 187658 T318 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49238_
[junit4:junit4] 2> 187661 T318 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49238_
[junit4:junit4] 2> 187669 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 187675 T393 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 187675 T393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 187686 T340 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 187699 T376 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 187700 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 187712 T360 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 187713 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 187727 T411 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 187738 T347 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 187738 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 187741 T412 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 187741 T412 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 187750 T412 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> ASYNC NEW_CORE C220 name=collection1 org.apache.solr.core.SolrCore@33f5c786 url=http://127.0.0.1:49231/collection1 node=127.0.0.1:49231_ C220_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:49231_, base_url=http://127.0.0.1:49231}
[junit4:junit4] 2> 188712 T397 C220 P49231 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49224/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 188713 T397 C220 P49231 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49231 START replicas=[http://127.0.0.1:49224/collection1/] nUpdates=100
[junit4:junit4] 2> 188715 T397 C220 P49231 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 188715 T397 C220 P49231 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 188715 T397 C220 P49231 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 188716 T397 C220 P49231 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 188716 T397 C220 P49231 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 188718 T397 C220 P49231 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49224/collection1/. core=collection1
[junit4:junit4] 2> 188718 T397 C220 P49231 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C221 name=collection1 org.apache.solr.core.SolrCore@623ff96d url=http://127.0.0.1:49224/collection1 node=127.0.0.1:49224_ C221_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49224_, base_url=http://127.0.0.1:49224, leader=true}
[junit4:junit4] 2> 188754 T355 C221 P49224 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=2
[junit4:junit4] 2> 188759 T351 C221 P49224 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 188760 T351 C221 P49224 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 188760 T351 C221 P49224 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 188761 T351 C221 P49224 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 3
[junit4:junit4] 2> 188763 T397 C220 P49231 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 188764 T397 C220 P49231 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 188769 T354 C221 P49224 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 2> 188771 T397 C220 P49231 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 188771 T397 C220 P49231 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 188771 T397 C220 P49231 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 188772 T397 C220 P49231 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 188775 T397 C220 P49231 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 188930 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 188933 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49238_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49238"}
[junit4:junit4] 2> 188933 T341 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=1
[junit4:junit4] 2> 188934 T341 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
[junit4:junit4] 2> 188948 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node3",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49231_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49231"}
[junit4:junit4] 2> 188968 T393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 188968 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 188970 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 188981 T411 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 188983 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 188971 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 189759 T412 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
[junit4:junit4] 2> 189760 T412 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty4-1372438112830/collection1
[junit4:junit4] 2> 189760 T412 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 189762 T412 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 189762 T412 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 189767 T412 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty4-1372438112830/collection1/'
[junit4:junit4] 2> 189769 T412 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.AliasIntegrationTest-jetty4-1372438112830/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 189769 T412 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.AliasIntegrationTest-jetty4-1372438112830/collection1/lib/README' to classloader
[junit4:junit4] 2> 189894 T412 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 190034 T412 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 190044 T412 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 190062 T412 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 191130 T412 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 191160 T412 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 191164 T412 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 191323 T412 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 191333 T412 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 191363 T412 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 191396 T412 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 191397 T412 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 191397 T412 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 191405 T412 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 191406 T412 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 191407 T412 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 191407 T412 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty4-1372438112830/collection1/, dataDir=./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty4/
[junit4:junit4] 2> 191408 T412 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44385e76
[junit4:junit4] 2> 191410 T412 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty4
[junit4:junit4] 2> 191411 T412 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty4/index/
[junit4:junit4] 2> 191411 T412 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 191412 T412 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.AliasIntegrationTest-1372438089620/jetty4/index
[junit4:junit4] 2> 191417 T412 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@31cfbd76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@21602963),segFN=segments_1,generation=1}
[junit4:junit4] 2> 191417 T412 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 191421 T412 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 191422 T412 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 191423 T412 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 191424 T412 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 191426 T412 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 191426 T412 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 191428 T412 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 191429 T412 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 191429 T412 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 191444 T412 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 191454 T412 oass.SolrIndexSearcher.<init> Opening Searcher@5f9aa8db main
[junit4:junit4] 2> 191455 T412 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 191456 T412 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 191464 T414 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5f9aa8db main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 191469 T412 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 191469 T412 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:49238 collection:collection1 shard:shard1
[junit4:junit4] 2> 191478 T412 oasc.ZkController.register We are http://127.0.0.1:49238/collection1/ and leader is http://127.0.0.1:49224/collection1/
[junit4:junit4] 2> 191479 T412 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:49238
[junit4:junit4] 2> 191479 T412 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 191480 T412 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C222 name=collection1 org.apache.solr.core.SolrCore@73fb57df url=http://127.0.0.1:49238/collection1 node=127.0.0.1:49238_ C222_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, node_name=127.0.0.1:49238_, base_url=http://127.0.0.1:49238}
[junit4:junit4] 2> 191481 T415 C222 P49238 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 191482 T415 C222 P49238 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 191482 T415 C222 P49238 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 191483 T415 C222 P49238 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 191500 T412 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 191500 T415 C222 P49238 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 191530 T318 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 191531 T318 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 191531 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 191567 T318 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 191569 T352 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 191572 T318 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 191573 T318 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 191575 T318 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 192088 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 192091 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49238_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49238"}
[junit4:junit4] 2> 192106 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 192106 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 192111 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 192113 T411 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 192114 T393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 192115 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 192571 T352 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 2> 192572 T352 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=core_node4&state=recovering&nodeName=127.0.0.1:49238_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1003
[junit4:junit4] 2> 192583 T318 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 193588 T318 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C223 name=collection1 org.apache.solr.core.SolrCore@73fb57df url=http://127.0.0.1:49238/collection1 node=127.0.0.1:49238_ C223_STATE=coll:collection1 core:collection1 props:{state=recovering, core=collection1, node_name=127.0.0.1:49238_, base_url=http://127.0.0.1:49238}
[junit4:junit4] 2> 194574 T415 C223 P49238 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:49224/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 194575 T415 C223 P49238 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:49238 START replicas=[http://127.0.0.1:49224/collection1/] nUpdates=100
[junit4:junit4] 2> 194576 T415 C223 P49238 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 194576 T415 C223 P49238 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 194577 T415 C223 P49238 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 194577 T415 C223 P49238 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 194577 T415 C223 P49238 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 194579 T415 C223 P49238 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:49224/collection1/. core=collection1
[junit4:junit4] 2> 194580 T415 C223 P49238 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 194604 T318 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> ASYNC NEW_CORE C224 name=collection1 org.apache.solr.core.SolrCore@623ff96d url=http://127.0.0.1:49224/collection1 node=127.0.0.1:49224_ C224_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49224_, base_url=http://127.0.0.1:49224, leader=true}
[junit4:junit4] 2> 194610 T352 C224 P49224 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 194620 T353 C224 P49224 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 194621 T353 C224 P49224 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 194621 T353 C224 P49224 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 194622 T353 C224 P49224 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 2
[junit4:junit4] 2> 194624 T415 C223 P49238 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 194624 T415 C223 P49238 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 194631 T356 C224 P49224 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 194632 T415 C223 P49238 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 194633 T415 C223 P49238 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 194633 T415 C223 P49238 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 194633 T415 C223 P49238 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 194637 T415 C223 P49238 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 195149 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 195151 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node4",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49238_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49238"}
[junit4:junit4] 2> 195166 T393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 195167 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 195168 T411 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 195168 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 195169 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 195168 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 195620 T318 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 195626 T318 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C225 name=collection1 org.apache.solr.core.SolrCore@1fd422e5 url=http://127.0.0.1:49220/collection1 node=127.0.0.1:49220_ C225_STATE=coll:control_collection core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49220_, base_url=http://127.0.0.1:49220, leader=true}
[junit4:junit4] 2> 195678 T331 C225 P49220 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 195682 T331 C225 P49220 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@67c860e2 lockFactory=org.apache.lucene.store.NativeFSLockFactory@69eb2cdb),segFN=segments_1,generation=1}
[junit4:junit4] 2> 195683 T331 C225 P49220 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 195685 T331 C225 P49220 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 195685 T331 C225 P49220 oass.SolrIndexSearcher.<init> Opening Searcher@415c7850 main
[junit4:junit4] 2> 195686 T331 C225 P49220 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 195688 T344 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@415c7850 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 195691 T331 C225 P49220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 13
[junit4:junit4] 2> C223_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49238_, base_url=http://127.0.0.1:49238}
[junit4:junit4] 2> 195759 T402 C223 P49238 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:49224/collection1/, StdNode: http://127.0.0.1:49227/collection1/, StdNode: http://127.0.0.1:49231/collection1/, StdNode: http://127.0.0.1:49238/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 195811 T403 C223 P49238 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 195813 T353 C224 P49224 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 195814 T353 C224 P49224 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 195814 T353 C224 P49224 oass.SolrIndexSearcher.<init> Opening Searcher@3bb4be09 main
[junit4:junit4] 2> 195815 T353 C224 P49224 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 195829 T362 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3bb4be09 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 195830 T353 C224 P49224 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 17
[junit4:junit4] 2> 195834 T403 C223 P49238 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@31cfbd76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@21602963),segFN=segments_1,generation=1}
[junit4:junit4] 2> ASYNC NEW_CORE C226 name=collection1 org.apache.solr.core.SolrCore@3076f54c url=http://127.0.0.1:49227/collection1 node=127.0.0.1:49227_ C226_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49227_, base_url=http://127.0.0.1:49227}
[junit4:junit4] 2> 195846 T368 C226 P49227 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C227 name=collection1 org.apache.solr.core.SolrCore@33f5c786 url=http://127.0.0.1:49231/collection1 node=127.0.0.1:49231_ C227_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49231_, base_url=http://127.0.0.1:49231}
[junit4:junit4] 2> 195848 T389 C227 P49231 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 195853 T368 C226 P49227 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4436f972 lockFactory=org.apache.lucene.store.NativeFSLockFactory@63b276e),segFN=segments_1,generation=1}
[junit4:junit4] 2> 195855 T389 C227 P49231 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@447088aa lockFactory=org.apache.lucene.store.NativeFSLockFactory@4472abe8),segFN=segments_1,generation=1}
[junit4:junit4] 2> 195856 T403 C223 P49238 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 195857 T403 C223 P49238 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 195858 T368 C226 P49227 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 195858 T368 C226 P49227 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 195860 T368 C226 P49227 oass.SolrIndexSearcher.<init> Opening Searcher@42296d15 main
[junit4:junit4] 2> 195860 T368 C226 P49227 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 195861 T389 C227 P49231 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 195861 T389 C227 P49231 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
[junit4:junit4] 2> 195874 T403 C223 P49238 oass.SolrIndexSearcher.<init> Opening Searcher@33f85c21 main
[junit4:junit4] 2> 195876 T403 C223 P49238 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 195864 T378 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@42296d15 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 195878 T414 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@33f85c21 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 195889 T368 C226 P49227 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 43
[junit4:junit4] 2> 195889 T403 C223 P49238 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 78
[junit4:junit4] 2> 195893 T389 C227 P49231 oass.SolrIndexSearcher.<init> Opening Searcher@68a33067 main
[junit4:junit4] 2> 195895 T389 C227 P49231 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 195896 T396 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@68a33067 main{StandardDirectoryReader(segments_1:1:nrt)}
[junit4:junit4] 2> 195896 T389 C227 P49231 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 48
[junit4:junit4] 2> 195898 T402 C223 P49238 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 147
[junit4:junit4] 2> 195915 T318 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 195922 T356 C224 P49224 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 195936 T369 C226 P49227 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 195960 T384 C227 P49231 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 195973 T404 C223 P49238 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 198000 T331 C225 P49220 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1439105679466954752)} 0 11
[junit4:junit4] 2> 198082 T368 C226 P49227 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439105679533015040&update.from=http://127.0.0.1:49224/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439105679533015040)} 0 10
[junit4:junit4] 2> 198106 T405 C223 P49238 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439105679533015040&update.from=http://127.0.0.1:49224/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439105679533015040)} 0 18
[junit4:junit4] 2> 198113 T385 C227 P49231 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1439105679533015040&update.from=http://127.0.0.1:49224/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1439105679533015040)} 0 4
[junit4:junit4] 2> 198118 T356 C224 P49224 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1439105679533015040)} 0 58
[junit4:junit4] 2> 198121 T404 C223 P49238 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 84
[junit4:junit4] 2> 198128 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 198258 T370 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&maxShardsPerNode=10&name=collection2&replicationFactor=1&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 198291 T340 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 198301 T342 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "maxShardsPerNode":"10",
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"collection2",
[junit4:junit4] 2> "replicationFactor":"1"}
[junit4:junit4] 2> 198302 T342 oasc.OverseerCollectionProcessor.createCollection Creating shard collection2_shard1_replica1 as part of slice shard1 of collection collection2 on 127.0.0.1:49224_
[junit4:junit4] 2> 198303 T342 oasc.OverseerCollectionProcessor.createCollection Creating shard collection2_shard2_replica1 as part of slice shard2 of collection collection2 on 127.0.0.1:49220_
[junit4:junit4] 2> 198312 T355 oasc.ZkController.publish publishing core=collection2_shard1_replica1 state=down
[junit4:junit4] 2> 198328 T333 oasc.ZkController.publish publishing core=collection2_shard2_replica1 state=down
[junit4:junit4] 2> 198337 T355 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 198340 T333 oasc.ZkController.waitForCoreNodeName look for our core node name
[junit4:junit4] 2> 199717 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 199720 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection2_shard1_replica1",
[junit4:junit4] 2> "collection":"collection2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49224_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49224"}
[junit4:junit4] 2> 199720 T341 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection2 with numShards 2
[junit4:junit4] 2> 199731 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection2_shard2_replica1",
[junit4:junit4] 2> "collection":"collection2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49220_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49220"}
[junit4:junit4] 2> 199745 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 199745 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 199746 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 199769 T393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 199775 T411 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 199776 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 200342 T355 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection2_shard1_replica1
[junit4:junit4] 2> 200343 T355 oasc.CoreContainer.create Creating SolrCore 'collection2_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection2_shard1_replica1
[junit4:junit4] 2> 200344 T355 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection2
[junit4:junit4] 2> 200345 T333 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection2_shard2_replica1
[junit4:junit4] 2> 200345 T333 oasc.CoreContainer.create Creating SolrCore 'collection2_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection2_shard2_replica1
[junit4:junit4] 2> 200346 T333 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection2
[junit4:junit4] 2> 200349 T355 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:collection2
[junit4:junit4] 2> 200350 T355 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 200355 T355 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 200356 T355 oascc.SolrZkClient.makePath makePath: /collections/collection2
[junit4:junit4] 2> 200357 T333 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:collection2
[junit4:junit4] 2> 200358 T333 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 200361 T333 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 200362 T333 oascc.SolrZkClient.makePath makePath: /collections/collection2
[junit4:junit4] 2> 200377 T333 oasc.ZkController.readConfigName Load collection config from:/collections/collection2
[junit4:junit4] 2> 200381 T355 oasc.ZkController.readConfigName Load collection config from:/collections/collection2
[junit4:junit4] 2> 200385 T333 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection2_shard2_replica1/'
[junit4:junit4] 2> 200385 T355 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection2_shard1_replica1/'
[junit4:junit4] 2> 200526 T355 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 200571 T333 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 200733 T355 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 200737 T333 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 200740 T355 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 200751 T355 oass.IndexSchema.readSchema [collection2_shard1_replica1] Schema name=test
[junit4:junit4] 2> 200782 T333 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 200794 T333 oass.IndexSchema.readSchema [collection2_shard2_replica1] Schema name=test
[junit4:junit4] 2> 202354 T355 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 202387 T355 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 202394 T355 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 202422 T333 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 202443 T333 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 202456 T333 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 202525 T333 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 202527 T355 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 202535 T333 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 202536 T355 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 202548 T355 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 202570 T355 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 202571 T355 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 202574 T355 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 202581 T355 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 202582 T355 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 202582 T355 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 202583 T355 oasc.SolrCore.<init> [collection2_shard1_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection2_shard1_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection2_shard1_replica1/data/
[junit4:junit4] 2> 202584 T355 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44385e76
[junit4:junit4] 2> 202585 T355 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection2_shard1_replica1/data
[junit4:junit4] 2> 202586 T355 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection2_shard1_replica1/data/index/
[junit4:junit4] 2> 202586 T355 oasc.SolrCore.initIndex WARN [collection2_shard1_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection2_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 202588 T355 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-jetty1-1372438096041/collection2_shard1_replica1/data/index
[junit4:junit4] 2> 202592 T333 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 202597 T355 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29b586fd lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ff0678c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 202598 T355 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 202615 T355 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 202616 T355 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 202617 T355 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 202619 T333 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 202619 T333 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 202620 T333 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 202621 T355 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 202622 T355 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 202622 T355 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 202623 T355 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 202624 T355 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 202625 T355 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 202657 T333 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 202658 T333 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 202658 T333 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 202659 T333 oasc.SolrCore.<init> [collection2_shard2_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection2_shard2_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection2_shard2_replica1/data/
[junit4:junit4] 2> 202664 T355 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 202665 T333 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@44385e76
[junit4:junit4] 2> 202685 T355 oass.SolrIndexSearcher.<init> Opening Searcher@34cc2998 main
[junit4:junit4] 2> 202691 T355 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 202691 T355 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 202697 T427 oasc.SolrCore.registerSearcher [collection2_shard1_replica1] Registered new searcher Searcher@34cc2998 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 202689 T333 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection2_shard2_replica1/data
[junit4:junit4] 2> 202700 T355 oasc.CoreContainer.registerCore registering core: collection2_shard1_replica1
[junit4:junit4] 2> 202700 T355 oasc.ZkController.register Register replica - core:collection2_shard1_replica1 address:http://127.0.0.1:49224 collection:collection2 shard:shard1
[junit4:junit4] 2> 202703 T333 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection2_shard2_replica1/data/index/
[junit4:junit4] 2> 202707 T333 oasc.SolrCore.initIndex WARN [collection2_shard2_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection2_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 202708 T355 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/shard1/election
[junit4:junit4] 2> 202708 T333 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.AliasIntegrationTest-controljetty-1372438089886/collection2_shard2_replica1/data/index
[junit4:junit4] 2> 202717 T333 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7ead8de4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a2667e0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 202718 T333 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 202723 T333 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 202724 T333 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 202725 T333 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 202727 T333 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 202728 T333 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 202729 T333 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 202729 T333 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 202730 T333 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 202732 T333 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 202733 T355 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
[junit4:junit4] 2> 202754 T333 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 202763 T333 oass.SolrIndexSearcher.<init> Opening Searcher@6b583cc9 main
[junit4:junit4] 2> 202765 T333 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 202765 T333 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 202772 T428 oasc.SolrCore.registerSearcher [collection2_shard2_replica1] Registered new searcher Searcher@6b583cc9 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 202778 T333 oasc.CoreContainer.registerCore registering core: collection2_shard2_replica1
[junit4:junit4] 2> 202782 T333 oasc.ZkController.register Register replica - core:collection2_shard2_replica1 address:http://127.0.0.1:49220 collection:collection2 shard:shard2
[junit4:junit4] 2> 202785 T333 oascc.SolrZkClient.makePath makePath: /collections/collection2/leader_elect/shard2/election
[junit4:junit4] 2> 202789 T355 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 202789 T355 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 202790 T355 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49224/collection2_shard1_replica1/
[junit4:junit4] 2> 202791 T355 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 202791 T355 oasc.SyncStrategy.syncToMe http://127.0.0.1:49224/collection2_shard1_replica1/ has no replicas
[junit4:junit4] 2> 202791 T355 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49224/collection2_shard1_replica1/ shard1
[junit4:junit4] 2> 202792 T355 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/shard1
[junit4:junit4] 2> 202816 T333 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
[junit4:junit4] 2> 202829 T333 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 202830 T333 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 202830 T333 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49220/collection2_shard2_replica1/
[junit4:junit4] 2> 202831 T333 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 202836 T333 oasc.SyncStrategy.syncToMe http://127.0.0.1:49220/collection2_shard2_replica1/ has no replicas
[junit4:junit4] 2> 202836 T333 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49220/collection2_shard2_replica1/ shard2
[junit4:junit4] 2> 202837 T333 oascc.SolrZkClient.makePath makePath: /collections/collection2/leaders/shard2
[junit4:junit4] 2> 202859 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 202916 T393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 202917 T411 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 202920 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 202921 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 202921 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 202924 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 202984 T355 oasc.ZkController.register We are http://127.0.0.1:49224/collection2_shard1_replica1/ and leader is http://127.0.0.1:49224/collection2_shard1_replica1/
[junit4:junit4] 2> 202985 T355 oasc.ZkController.register No LogReplay needed for core=collection2_shard1_replica1 baseURL=http://127.0.0.1:49224
[junit4:junit4] 2> 202985 T355 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 202987 T355 oasc.ZkController.publish publishing core=collection2_shard1_replica1 state=active
[junit4:junit4] 2> 202989 T333 oasc.ZkController.register We are http://127.0.0.1:49220/collection2_shard2_replica1/ and leader is http://127.0.0.1:49220/collection2_shard2_replica1/
[junit4:junit4] 2> 202990 T333 oasc.ZkController.register No LogReplay needed for core=collection2_shard2_replica1 baseURL=http://127.0.0.1:49220
[junit4:junit4] 2> 202991 T333 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 202991 T333 oasc.ZkController.publish publishing core=collection2_shard2_replica1 state=active
[junit4:junit4] 2> 202997 T355 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 202999 T333 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 203002 T355 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=collection2_shard1_replica1&action=CREATE&collection=collection2&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=4690
[junit4:junit4] 2> 203003 T333 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=collection2_shard2_replica1&action=CREATE&collection=collection2&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=4675
[junit4:junit4] 2> 203007 T342 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: collection2
[junit4:junit4] 2> 203074 T376 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4] 2> 203088 T370 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=2&maxShardsPerNode=10&name=collection2&replicationFactor=1&action=CREATE&wt=javabin&version=2} status=0 QTime=4841
[junit4:junit4] 2> 203090 T318 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 203092 T318 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 203093 T340 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 203099 T430 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cebdec4 name:ZooKeeperConnection Watcher:127.0.0.1:49217/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 203099 T318 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 203110 T318 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 203111 T342 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=4690},core=collection2_shard1_replica1},null={responseHeader={status=0,QTime=4675},core=collection2_shard2_replica1}}}
[junit4:junit4] 2> 203119 T318 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection2 failOnTimeout:true timeout (sec):330
[junit4:junit4] 2> 203121 T318 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 204127 T318 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 204464 T341 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 204467 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection2_shard1_replica1",
[junit4:junit4] 2> "collection":"collection2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49224_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49224"}
[junit4:junit4] 2> 204478 T341 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"core_node2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection2_shard2_replica1",
[junit4:junit4] 2> "collection":"collection2",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49220_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49220"}
[junit4:junit4] 2> 204494 T376 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 204495 T340 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 204495 T393 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 204496 T347 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 204498 T430 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 204500 T411 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 204501 T360 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 205131 T318 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 205134 T318 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection2
[junit4:junit4] 2> ASYNC NEW_CORE C228 name=collection1 org.apache.solr.core.SolrCore@33f5c786 url=http://127.0.0.1:49231/collection1 node=127.0.0.1:49231_ C228_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49231_, base_url=http://127.0.0.1:49231}
[junit4:junit4] 2> 205197 T384 C228 P49231 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49224/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1439105686975807488)]} 0 15
[junit4:junit4] 2> ASYNC NEW_CORE C229 name=collection1 org.apache.solr.core.SolrCore@3076f54c url=http://127.0.0.1:49227/collection1 node=127.0.0.1:49227_ C229_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49227_, base_url=http://127.0.0.1:49227}
[junit4:junit4] 2> 205198 T370 C229 P49227 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49224/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1439105686975807488)]} 0 6
[junit4:junit4] 2> ASYNC NEW_CORE C230 name=collection1 org.apache.solr.core.SolrCore@73fb57df url=http://127.0.0.1:49238/collection1 node=127.0.0.1:49238_ C230_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49238_, base_url=http://127.0.0.1:49238}
[junit4:junit4] 2> 205198 T406 C230 P49238 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49224/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1439105686975807488)]} 0 8
[junit4:junit4] 2> ASYNC NEW_CORE C231 name=collection1 org.apache.solr.core.SolrCore@623ff96d url=http://127.0.0.1:49224/collection1 node=127.0.0.1:49224_ C231_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49224_, base_url=http://127.0.0.1:49224, leader=true}
[junit4:junit4] 2> 205231 T355 C231 P49224 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49238/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1439105686975807488)]} 0 73
[junit4:junit4] 2> 205233 T404 C230 P49238 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6]} 0 95
[junit4:junit4] 2> 205263 T407 C230 P49238 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49224/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1439105687071227904)]} 0 3
[junit4:junit4] 2> 205274 T367 C229 P49227 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49224/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1439105687071227904)]} 0 14
[junit4:junit4] 2> 205276 T385 C228 P49231 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49224/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1439105687071227904)]} 0 7
[junit4:junit4] 2> 205304 T352 C231 P49224 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1439105687071227904)]} 0 56
[junit4:junit4] 2> 205325 T386 C228 P49231 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49224/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1439105687135191040)]} 0 2
[junit4:junit4] 2> 205330 T368 C229 P49227 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49224/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1439105687135191040)]} 0 2
[junit4:junit4] 2> 205334 T403 C230 P49238 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:49224/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1439105687135191040)]} 0 2
[junit4:junit4] 2> 205344 T353 C231 P49224 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1439105687135191040)]} 0 34
[junit4:junit4] 2> 205350 T369 C229 P49227 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:49224/collection1/, StdNode: http://127.0.0.1:49227/collection1/, StdNode: http://127.0.0.1:49231/collection1/, StdNode: http://127.0.0.1:49238/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 205388 T387 C228 P49231 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C232 name=collection1 org.apache.solr.core.SolrCore@3076f54c url=http://127.0.0.1:49227/collection1 node=127.0.0.1:49227_ C232_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49227_, base_url=http://127.0.0.1:49227}
[junit4:junit4] 2> 205412 T371 C232 P49227 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C233 name=collection1 org.apache.solr.core.SolrCore@73fb57df url=http://127.0.0.1:49238/collection1 node=127.0.0.1:49238_ C233_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49238_, base_url=http://127.0.0.1:49238}
[junit4:junit4] 2> 205414 T402 C233 P49238 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C234 name=collection1 org.apache.solr.core.SolrCore@623ff96d url=http://127.0.0.1:49224/collection1 node=127.0.0.1:49224_ C234_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49224_, base_url=http://127.0.0.1:49224, leader=true}
[junit4:junit4] 2> 205415 T354 C234 P49224 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> ASYNC NEW_CORE C235 name=collection1 org.apache.solr.core.SolrCore@33f5c786 url=http://127.0.0.1:49231/collection1 node=127.0.0.1:49231_ C235_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49231_, base_url=http://127.0.0.1:49231}
[junit4:junit4] 2> 205660 T387 C235 P49231 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@447088aa lockFactory=org.apache.lucene.store.NativeFSLockFactory@4472abe8),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@447088aa lockFactory=org.apache.lucene.store.NativeFSLockFactory@4472abe8),segFN=segments_2,generation=2}
[junit4:junit4] 2> 205661 T387 C235 P49231 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 205681 T402 C233 P49238 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@31cfbd76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@21602963),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@31cfbd76 lockFactory=org.apache.lucene.store.NativeFSLockFactory@21602963),segFN=segments_2,generation=2}
[junit4:junit4] 2> 205682 T402 C233 P49238 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 205801 T402 C233 P49238 oass.SolrIndexSearcher.<init> Opening Searcher@35acd991 main
[junit4:junit4] 2> 205802 T402 C233 P49238 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 205803 T414 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@35acd991 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C3)}
[junit4:junit4] 2> 205813 T402 C233 P49238 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 400
[junit4:junit4] 2> 205831 T387 C235 P49231 oass.SolrIndexSearcher.<init> Opening Searcher@16a6cf56 main
[junit4:junit4] 2> 205834 T396 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@16a6cf56 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C3)}
[junit4:junit4] 2> 205835 T387 C235 P49231 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 205835 T387 C235 P49231 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 447
[junit4:junit4] 2> 205849 T371 C232 P49227 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4436f972 lockFactory=org.apache.lucene.store.NativeFSLockFactory@63b276e),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4436f972 lockFactory=org.apache.lucene.store.NativeFSLockFactory@63b276e),segFN=segments_2,generation=2}
[junit4:junit4] 2> 205850 T371 C232 P49227 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 205859 T354 C234 P49224 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1672f0a6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18bdefd7),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1672f0a6 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18bdefd7),segFN=segments_2,generation=2}
[junit4:junit4] 2> 205865 T354 C234 P49224 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 205891 T371 C232 P49227 oass.SolrIndexSearcher.<init> Opening Searcher@16041baf main
[junit4:junit4] 2> 205899 T371 C232 P49227 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 205901 T378 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@16041baf main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C3)}
[junit4:junit4] 2> 205907 T371 C232 P49227 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 495
[junit4:junit4] 2> 205923 T354 C234 P49224 oass.SolrIndexSearcher.<init> Opening Searcher@5de5ffa4 main
[junit4:junit4] 2> 205927 T362 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@5de5ffa4 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C3)}
[junit4:junit4] 2> 205930 T354 C234 P49224 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 205930 T354 C234 P49224 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 516
[junit4:junit4] 2> 205932 T369 C232 P49227 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 583
[junit4:junit4] 2> ASYNC NEW_CORE C236 name=collection2_shard2_replica1 org.apache.solr.core.SolrCore@1ca3acfe url=http://127.0.0.1:49220/collection2_shard2_replica1 node=127.0.0.1:49220_ C236_STATE=coll:collection2 core:collection2_shard2_replica1 props:{state=active, core=collection2_shard2_replica1, node_name=127.0.0.1:49220_, base_url=http://127.0.0.1:49220, leader=true}
[junit4:junit4] 2> 205970 T335 C236 P49220 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7ead8de4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a2667e0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 205972 T335 C236 P49220 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 205992 T335 C236 P49220 oasup.LogUpdateProcessor.finish [collection2_shard2_replica1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1439105687801036800)]} 0 47
[junit4:junit4] 2> ASYNC NEW_CORE C237 name=collection2_shard1_replica1 org.apache.solr.core.SolrCore@162382a5 url=http://127.0.0.1:49224/collection2_shard1_replica1 node=127.0.0.1:49224_ C237_STATE=coll:collection2 core:collection2_shard1_replica1 props:{state=active, core=collection2_shard1_replica1, node_name=127.0.0.1:49224_, base_url=http://127.0.0.1:49224, leader=true}
[junit4:junit4] 2> 206061 T351 C237 P49224 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29b586fd lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ff0678c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 206062 T351 C237 P49224 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 206070 T351 C237 P49224 oasup.LogUpdateProcessor.finish [collection2_shard1_replica1] webapp= path=/update params={distrib.from=http://127.0.0.1:49220/collection2_shard2_replica1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1439105687871291392)]} 0 59
[junit4:junit4] 2> 206072 T336 C236 P49220 oasup.LogUpdateProcessor.finish [collection2_shard2_replica1] webapp= path=/update params={wt=javabin&version=2} {add=[10]} 0 76
[junit4:junit4] 2> 206076 T355 C237 P49224 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:49224/collection2_shard1_replica1/, StdNode: http://127.0.0.1:49220/collection2_shard2_replica1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 206093 T352 C237 P49224 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 206095 T331 C236 P49220 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 206357 T352 C237 P49224 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29b586fd lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ff0678c),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@29b586fd lockFactory=org.apache.lucene.store.NativeFSLockFactory@4ff0678c),segFN=segments_2,generation=2}
[junit4:junit4] 2> 206357 T352 C237 P49224 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 206359 T331 C236 P49220 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7ead8de4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a2667e0),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7ead8de4 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4a2667e0),segFN=segments_2,generation=2}
[junit4:junit4] 2> 206360 T331 C236 P49220 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 206368 T352 C237 P49224 oass.SolrIndexSearcher.<init> Opening Searcher@47111447 main
[junit4:junit4] 2> 206376 T427 oasc.SolrCore.registerSearcher [collection2_shard1_replica1] Registered new searcher Searcher@47111447 main{StandardDirectoryReader(segments_2:3:nrt _0(5.0):C1)}
[junit4:junit4] 2> 206403 T352 C237 P49224 oasu.DirectUpdateHandler2.commit end_commit_flush
[
[...truncated too long message...]
ive",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49504_rovu",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49504/rovu"}
[junit4:junit4] 2> 619076 T1176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 619094 T1085 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 49508
[junit4:junit4] 2> 619095 T1085 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1760165217
[junit4:junit4] 2> 620104 T1085 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 620105 T1085 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 620109 T1085 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@59753857
[junit4:junit4] 2> 620125 T1085 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=2,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=228,adds=228,deletesById=115,deletesByQuery=0,errors=0,cumulative_adds=270,cumulative_deletesById=115,cumulative_deletesByQuery=1,cumulative_errors=0}
[junit4:junit4] 2> 620126 T1085 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
[junit4:junit4] 2> 620127 T1085 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 2> 620127 T1085 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
[junit4:junit4] 2> ASYNC NEW_CORE C626 name=collection1 org.apache.solr.core.SolrCore@59753857 url=http://127.0.0.1:49508/rovu/collection1 node=127.0.0.1:49508_rovu C626_STATE=coll:collection1 core:collection1 props:{state=active, core=collection1, node_name=127.0.0.1:49508_rovu, base_url=http://127.0.0.1:49508/rovu, leader=true}
[junit4:junit4] 2> 620234 T1085 C626 P49508 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f2e88b1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@605f6f30),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1f2e88b1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@605f6f30),segFN=segments_3,generation=3}
[junit4:junit4] 2> 620235 T1085 C626 P49508 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 620253 T1085 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
[junit4:junit4] 2> 620254 T1085 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 2> 620254 T1085 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1372438453439/jetty4 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1372438453439/jetty4;done=false>>]
[junit4:junit4] 2> 620255 T1085 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1372438453439/jetty4
[junit4:junit4] 2> 620262 T1085 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ShardSplitTest-1372438453439/jetty4/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ShardSplitTest-1372438453439/jetty4/index;done=false>>]
[junit4:junit4] 2> 620263 T1085 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ShardSplitTest-1372438453439/jetty4/index
[junit4:junit4] 2> 620263 T1159 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 620276 T1159 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 620277 T1159 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 620278 T1207 oasc.Overseer$ClusterStateUpdater.amILeader According to ZK I (id=89944126485233676-127.0.0.1:49508_rovu-n_0000000004) am no longer a leader.
[junit4:junit4] 2> 620286 T1176 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 620287 T1176 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 620287 T1176 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 620332 T1085 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/rovu,null}
[junit4:junit4] 2> 620418 T1085 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 620422 T1085 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:49491 49491
[junit4:junit4] 2> 620454 T1085 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 49494
[junit4:junit4] 2> !!!! WARNING: best effort to remove /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-1372438453439 FAILED !!!!!
[junit4:junit4] 2> 620455 T1085 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 620456 T1085 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:49491 49491
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=D6C4F79DCF3F97A1 -Dtests.slow=true -Dtests.locale=fr_LU -Dtests.timezone=Etc/GMT+2 -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 94.1s | ShardSplitTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:49498/rovu returned non ok status:500, message:Server Error
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([D6C4F79DCF3F97A1:57227985B860F79D]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:228)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:150)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:835)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:724)
[junit4:junit4] 2> 620465 T1085 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 94159 T1084 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {id=PostingsFormat(name=MockRandom), _version_=PostingsFormat(name=SimpleText), n_ti=PostingsFormat(name=Memory doPackFST= false)}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=fr_LU, timezone=Etc/GMT+2
[junit4:junit4] 2> NOTE: Mac OS X 10.8.4 x86_64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=2,threads=1,free=49230864,total=134307840
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TimeZoneUtilsTest, TestWriterPerf, TestClassNameShortening, TestBinaryResponseWriter, DirectUpdateHandlerTest, ZkSolrClientTest, DebugComponentTest, ShowFileRequestHandlerTest, SolrIndexSplitterTest, TestDistributedGrouping, SynonymTokenizerTest, CachingDirectoryFactoryTest, AliasIntegrationTest, MultiTermTest, HdfsRecoveryZkTest, TestSchemaSimilarityResource, TestPropInject, TestRTGBase, JSONWriterTest, TestBM25SimilarityFactory, TestElisionMultitermQuery, TestHashPartitioner, PrimitiveFieldTypeTest, DOMUtilTest, TestSolrJ, BinaryUpdateRequestHandlerTest, TestSort, CircularListTest, TestReload, TestSolrQueryParser, HdfsChaosMonkeySafeLeaderTest, TestJoin, TestSerializedLuceneMatchVersion, TestCodecSupport, DisMaxRequestHandlerTest, SimplePostToolTest, OutputWriterTest, DocumentBuilderTest, ShardRoutingTest, TestMultiCoreConfBootstrap, TestReversedWildcardFilterFactory, IndexBasedSpellCheckerTest, AlternateDirectoryTest, CoreContainerCoreInitFailuresTest, ShardSplitTest]
[junit4:junit4] Completed in 94.72s, 1 test, 1 error <<< FAILURES!
[...truncated 845 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:386: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:366: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:190: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:443: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1245: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:889: There were test failures: 312 suites, 1316 tests, 2 errors, 25 ignored (12 assumptions)
Total time: 105 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:-UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure