You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/05/23 09:19:35 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_21) - Build #
5789 - Failure!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/5789/
Java: 32bit/jdk1.7.0_21 -server -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
There are still nodes recoverying - waited for 230 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
at __randomizedtesting.SeedInfo.seed([437E8AC0DD05609D:C29804D8AA5A00A1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9659 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4:junit4] 2> 453728 T1674 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /du/
[junit4:junit4] 2> 453731 T1674 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-CollectionsAPIDistributedZkTest-1369292776192
[junit4:junit4] 2> 453731 T1674 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 453732 T1675 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 453832 T1674 oasc.ZkTestServer.run start zk server on port:36211
[junit4:junit4] 2> 453833 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 454412 T1681 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c6134c name:ZooKeeperConnection Watcher:127.0.0.1:36211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 454413 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 454413 T1674 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 454417 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 454418 T1683 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15adb15 name:ZooKeeperConnection Watcher:127.0.0.1:36211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 454419 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 454419 T1674 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 454421 T1674 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 454423 T1674 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 454424 T1674 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 454426 T1674 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 454426 T1674 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 454429 T1674 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 454430 T1674 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 454432 T1674 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 454432 T1674 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 454435 T1674 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 454435 T1674 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 454437 T1674 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 454438 T1674 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 454440 T1674 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 454441 T1674 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 454442 T1674 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 454443 T1674 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 454445 T1674 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 454446 T1674 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 454448 T1674 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 454448 T1674 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 454516 T1674 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 454520 T1674 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:48584
[junit4:junit4] 2> 454521 T1674 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 454521 T1674 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 454522 T1674 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912
[junit4:junit4] 2> 454522 T1674 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/solr.xml
[junit4:junit4] 2> 454522 T1674 oasc.CoreContainer.<init> New CoreContainer 32313319
[junit4:junit4] 2> 454523 T1674 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/'
[junit4:junit4] 2> 454523 T1674 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/'
[junit4:junit4] 2> 454572 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912
[junit4:junit4] 2> 454572 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/collection1
[junit4:junit4] 2> 454573 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/collection1/conf
[junit4:junit4] 2> 454578 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/collection1/conf/xslt
[junit4:junit4] 2> 454594 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/collection1/lib
[junit4:junit4] 2> 454595 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/collection1/lib/classes
[junit4:junit4] 2> 454596 T1674 oasc.SolrCoreDiscoverer.addCore Discovered properties file /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/conf/core.properties, adding to cores
[junit4:junit4] 2> 454598 T1674 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 454598 T1674 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 454599 T1674 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 454599 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 454599 T1674 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 454600 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 454600 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 454600 T1674 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 454601 T1674 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 454601 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 454604 T1674 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 454605 T1674 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36211/solr
[junit4:junit4] 2> 454605 T1674 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 454606 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 454607 T1694 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1eea404 name:ZooKeeperConnection Watcher:127.0.0.1:36211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 454607 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 454610 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 454613 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 454614 T1696 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@188b4cc name:ZooKeeperConnection Watcher:127.0.0.1:36211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 454614 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 454616 T1674 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 454618 T1674 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 454620 T1674 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 454621 T1674 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48584_du
[junit4:junit4] 2> 454622 T1674 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:48584_du
[junit4:junit4] 2> 454624 T1674 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 454628 T1674 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 454629 T1674 oasc.Overseer.start Overseer (id=89737971380649987-127.0.0.1:48584_du-n_0000000000) starting
[junit4:junit4] 2> 454631 T1674 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 454634 T1698 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 454634 T1674 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 454636 T1674 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 454637 T1674 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 454639 T1697 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 454640 T1674 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 454641 T1674 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 454641 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 454643 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 454644 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 454645 T1701 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18462b name:ZooKeeperConnection Watcher:127.0.0.1:36211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 454645 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 454646 T1674 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 454648 T1674 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 454708 T1674 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 454710 T1674 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:45310
[junit4:junit4] 2> 454710 T1674 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 454711 T1674 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 454711 T1674 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109
[junit4:junit4] 2> 454711 T1674 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/solr.xml
[junit4:junit4] 2> 454712 T1674 oasc.CoreContainer.<init> New CoreContainer 10170407
[junit4:junit4] 2> 454712 T1674 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/'
[junit4:junit4] 2> 454712 T1674 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/'
[junit4:junit4] 2> 454748 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109
[junit4:junit4] 2> 454748 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/collection1
[junit4:junit4] 2> 454749 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/collection1/conf
[junit4:junit4] 2> 454753 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/collection1/conf/xslt
[junit4:junit4] 2> 454768 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/collection1/lib
[junit4:junit4] 2> 454769 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/collection1/lib/classes
[junit4:junit4] 2> 454770 T1674 oasc.SolrCoreDiscoverer.addCore Discovered properties file /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/conf/core.properties, adding to cores
[junit4:junit4] 2> 454772 T1674 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 454772 T1674 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 454773 T1674 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 454773 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 454773 T1674 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 454774 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 454774 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 454774 T1674 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 454774 T1674 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 454775 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 454777 T1674 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 454778 T1674 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36211/solr
[junit4:junit4] 2> 454778 T1674 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 454779 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 454780 T1712 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@944cb9 name:ZooKeeperConnection Watcher:127.0.0.1:36211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 454781 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 454783 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 454785 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 454787 T1714 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@132b6c6 name:ZooKeeperConnection Watcher:127.0.0.1:36211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 454787 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 454790 T1674 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 455792 T1674 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45310_du
[junit4:junit4] 2> 455793 T1674 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45310_du
[junit4:junit4] 2> 455796 T1696 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 455797 T1714 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 455797 T1701 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 455800 T1674 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 455801 T1674 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 455801 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 455874 T1674 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 455876 T1674 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49820
[junit4:junit4] 2> 455877 T1674 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 455877 T1674 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 455877 T1674 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274
[junit4:junit4] 2> 455878 T1674 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/solr.xml
[junit4:junit4] 2> 455878 T1674 oasc.CoreContainer.<init> New CoreContainer 7006400
[junit4:junit4] 2> 455878 T1674 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/'
[junit4:junit4] 2> 455879 T1674 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/'
[junit4:junit4] 2> 455912 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274
[junit4:junit4] 2> 455913 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/collection1
[junit4:junit4] 2> 455913 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/collection1/conf
[junit4:junit4] 2> 455918 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/collection1/conf/xslt
[junit4:junit4] 2> 455932 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/collection1/lib
[junit4:junit4] 2> 455933 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/collection1/lib/classes
[junit4:junit4] 2> 455934 T1674 oasc.SolrCoreDiscoverer.addCore Discovered properties file /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/conf/core.properties, adding to cores
[junit4:junit4] 2> 455936 T1674 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 455936 T1674 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 455937 T1674 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 455937 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 455937 T1674 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 455937 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 455938 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 455938 T1674 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 455938 T1674 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 455939 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 455941 T1674 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 455942 T1674 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36211/solr
[junit4:junit4] 2> 455942 T1674 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 455943 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 455944 T1726 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@17a97c3 name:ZooKeeperConnection Watcher:127.0.0.1:36211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 455945 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 455946 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 455949 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 455950 T1728 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@509012 name:ZooKeeperConnection Watcher:127.0.0.1:36211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 455950 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 455953 T1674 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 456956 T1674 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49820_du
[junit4:junit4] 2> 456958 T1674 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49820_du
[junit4:junit4] 2> 456961 T1714 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 456961 T1701 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 456961 T1696 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 456961 T1728 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 456964 T1674 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 456965 T1674 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 456965 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 457028 T1674 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 457030 T1674 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37077
[junit4:junit4] 2> 457030 T1674 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 457030 T1674 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 457031 T1674 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428
[junit4:junit4] 2> 457031 T1674 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/solr.xml
[junit4:junit4] 2> 457031 T1674 oasc.CoreContainer.<init> New CoreContainer 2166123
[junit4:junit4] 2> 457032 T1674 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/'
[junit4:junit4] 2> 457032 T1674 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/'
[junit4:junit4] 2> 457080 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428
[junit4:junit4] 2> 457081 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/collection1
[junit4:junit4] 2> 457082 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/collection1/conf
[junit4:junit4] 2> 457089 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/collection1/conf/xslt
[junit4:junit4] 2> 457113 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/collection1/lib
[junit4:junit4] 2> 457114 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/collection1/lib/classes
[junit4:junit4] 2> 457116 T1674 oasc.SolrCoreDiscoverer.addCore Discovered properties file /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/conf/core.properties, adding to cores
[junit4:junit4] 2> 457118 T1674 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 457118 T1674 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 457119 T1674 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 457119 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 457119 T1674 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 457120 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 457120 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 457120 T1674 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 457121 T1674 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 457121 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 457124 T1674 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 457124 T1674 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36211/solr
[junit4:junit4] 2> 457125 T1674 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 457125 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 457127 T1740 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13030e9 name:ZooKeeperConnection Watcher:127.0.0.1:36211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 457127 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 457128 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 457131 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 457132 T1742 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dd8b02 name:ZooKeeperConnection Watcher:127.0.0.1:36211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 457132 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 457135 T1674 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 458137 T1674 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37077_du
[junit4:junit4] 2> 458139 T1674 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37077_du
[junit4:junit4] 2> 458141 T1696 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 458141 T1728 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 458141 T1714 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 458141 T1742 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 458141 T1701 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 458145 T1674 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 458145 T1674 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 458146 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 458207 T1674 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 458209 T1674 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:49168
[junit4:junit4] 2> 458209 T1674 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 458210 T1674 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 458210 T1674 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609
[junit4:junit4] 2> 458210 T1674 oasc.CoreContainer$Initializer.initialize looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/solr.xml
[junit4:junit4] 2> 458211 T1674 oasc.CoreContainer.<init> New CoreContainer 16498869
[junit4:junit4] 2> 458211 T1674 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/'
[junit4:junit4] 2> 458211 T1674 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/'
[junit4:junit4] 2> 458255 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609
[junit4:junit4] 2> 458256 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/collection1
[junit4:junit4] 2> 458256 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/collection1/conf
[junit4:junit4] 2> 458261 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/collection1/conf/xslt
[junit4:junit4] 2> 458275 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/collection1/lib
[junit4:junit4] 2> 458276 T1674 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/collection1/lib/classes
[junit4:junit4] 2> 458277 T1674 oasc.SolrCoreDiscoverer.addCore Discovered properties file /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/conf/core.properties, adding to cores
[junit4:junit4] 2> 458279 T1674 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 458279 T1674 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 458280 T1674 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 458280 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 458280 T1674 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 458280 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 458281 T1674 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 458281 T1674 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 458281 T1674 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 458282 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 458284 T1674 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 458285 T1674 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:36211/solr
[junit4:junit4] 2> 458285 T1674 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 458286 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 458287 T1754 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8edb5e name:ZooKeeperConnection Watcher:127.0.0.1:36211 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 458287 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 458289 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 458292 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 458293 T1756 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f45b2f name:ZooKeeperConnection Watcher:127.0.0.1:36211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 458293 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 458296 T1674 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 459299 T1674 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49168_du
[junit4:junit4] 2> 459300 T1674 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:49168_du
[junit4:junit4] 2> 459302 T1701 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 459303 T1714 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 459303 T1728 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 459302 T1696 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 459303 T1756 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 459303 T1742 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 459306 T1674 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 2> 459306 T1674 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 459307 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 459310 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 459314 T1708 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 459315 T1696 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 459316 T1698 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "maxShardsPerNode":null,
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"nodes_used_collection",
[junit4:junit4] 2> "replicationFactor":"2"}
[junit4:junit4] 2> 459317 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:37077_du
[junit4:junit4] 2> 459317 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:49168_du
[junit4:junit4] 2> 459317 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:49820_du
[junit4:junit4] 2> 459318 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:48584_du
[junit4:junit4] 2> 459319 T1721 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 459319 T1721 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 459319 T1690 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/nodes_used_collection_shard2_replica2
[junit4:junit4] 2> 459320 T1690 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 459320 T1738 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 459321 T1690 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 459320 T1750 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/nodes_used_collection_shard1_replica2
[junit4:junit4] 2> 459321 T1750 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 459321 T1690 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 459321 T1738 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 459320 T1721 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 459322 T1750 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 459323 T1690 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 459322 T1721 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 459322 T1738 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 459324 T1738 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 459323 T1690 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 459323 T1750 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 459325 T1738 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 459324 T1721 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 459325 T1738 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 459325 T1721 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 459326 T1690 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 459326 T1750 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 459327 T1750 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 459327 T1738 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 459327 T1721 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 459328 T1690 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/nodes_used_collection_shard2_replica2/'
[junit4:junit4] 2> 459330 T1738 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/nodes_used_collection_shard1_replica1/'
[junit4:junit4] 2> 459331 T1750 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 459331 T1721 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/nodes_used_collection_shard2_replica1/'
[junit4:junit4] 2> 459335 T1750 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/nodes_used_collection_shard1_replica2/'
[junit4:junit4] 2> 459377 T1750 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 459379 T1738 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 459380 T1690 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 459383 T1721 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 459428 T1750 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 459430 T1750 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 459433 T1721 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 459434 T1750 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica2] Schema name=test
[junit4:junit4] 2> 459435 T1721 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 459439 T1721 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica1] Schema name=test
[junit4:junit4] 2> 459444 T1738 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 459445 T1690 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 459446 T1738 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 459447 T1690 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 459465 T1738 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica1] Schema name=test
[junit4:junit4] 2> 459465 T1690 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica2] Schema name=test
[junit4:junit4] 2> 459920 T1750 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 459936 T1750 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 459939 T1750 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 459955 T1750 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 459959 T1750 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 459969 T1750 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 459970 T1750 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 459970 T1750 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 459971 T1750 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 459972 T1750 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 459973 T1750 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 459973 T1750 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 459973 T1750 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/nodes_used_collection_shard1_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/nodes_used_collection_shard1_replica2/data/
[junit4:junit4] 2> 459974 T1750 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@8bf924
[junit4:junit4] 2> 459975 T1750 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 459976 T1750 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/nodes_used_collection_shard1_replica2/data
[junit4:junit4] 2> 459976 T1750 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/nodes_used_collection_shard1_replica2/data/index/
[junit4:junit4] 2> 459976 T1750 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/nodes_used_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 459977 T1750 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/nodes_used_collection_shard1_replica2/data/index
[junit4:junit4] 2> 459980 T1750 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ed7cd7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f5df41),segFN=segments_1,generation=1}
[junit4:junit4] 2> 459980 T1750 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 459983 T1750 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 459983 T1750 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 459984 T1750 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 459985 T1750 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 459985 T1750 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 459986 T1750 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 459986 T1750 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 459987 T1750 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 459987 T1721 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 459987 T1750 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 459992 T1750 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 459995 T1721 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 459996 T1750 oass.SolrIndexSearcher.<init> Opening Searcher@165c214 main
[junit4:junit4] 2> 459997 T1750 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/nodes_used_collection_shard1_replica2/data/tlog
[junit4:junit4] 2> 459998 T1750 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 459998 T1750 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 459999 T1721 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 460008 T1762 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@165c214 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 460010 T1750 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=down
[junit4:junit4] 2> 460014 T1738 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 460018 T1721 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 460020 T1738 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 460022 T1721 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 460022 T1738 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 460025 T1721 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 460026 T1721 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 460027 T1721 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 460027 T1721 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 460028 T1721 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 460028 T1721 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 460028 T1721 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 460029 T1721 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/nodes_used_collection_shard2_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/nodes_used_collection_shard2_replica1/data/
[junit4:junit4] 2> 460030 T1721 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@8bf924
[junit4:junit4] 2> 460030 T1721 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 460031 T1721 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/nodes_used_collection_shard2_replica1/data
[junit4:junit4] 2> 460031 T1721 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/nodes_used_collection_shard2_replica1/data/index/
[junit4:junit4] 2> 460031 T1721 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/nodes_used_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 460032 T1721 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/nodes_used_collection_shard2_replica1/data/index
[junit4:junit4] 2> 460033 T1721 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@5e984c lockFactory=org.apache.lucene.store.NativeFSLockFactory@ce6c4c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 460034 T1738 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 460034 T1721 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 460036 T1721 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 460036 T1721 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 460036 T1721 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 460037 T1721 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 460037 T1721 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 460038 T1721 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 460038 T1738 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 460038 T1721 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 460039 T1721 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 460039 T1721 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 460041 T1738 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 460041 T1721 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 460042 T1738 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 460042 T1738 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 460042 T1738 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 460043 T1738 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 460043 T1721 oass.SolrIndexSearcher.<init> Opening Searcher@ebc316 main
[junit4:junit4] 2> 460043 T1738 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 460044 T1721 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/nodes_used_collection_shard2_replica1/data/tlog
[junit4:junit4] 2> 460044 T1738 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 460044 T1738 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/nodes_used_collection_shard1_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/nodes_used_collection_shard1_replica1/data/
[junit4:junit4] 2> 460044 T1721 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 460045 T1738 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@8bf924
[junit4:junit4] 2> 460045 T1721 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 460045 T1738 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 460046 T1738 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/nodes_used_collection_shard1_replica1/data
[junit4:junit4] 2> 460046 T1738 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/nodes_used_collection_shard1_replica1/data/index/
[junit4:junit4] 2> 460047 T1738 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/nodes_used_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 460047 T1738 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/nodes_used_collection_shard1_replica1/data/index
[junit4:junit4] 2> 460048 T1763 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@ebc316 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 460049 T1738 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4e021f lockFactory=org.apache.lucene.store.NativeFSLockFactory@f774be),segFN=segments_1,generation=1}
[junit4:junit4] 2> 460049 T1721 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=down
[junit4:junit4] 2> 460049 T1738 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 460051 T1738 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 460051 T1738 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 460052 T1738 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 460052 T1738 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 460052 T1738 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 460053 T1738 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 460053 T1690 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 460053 T1738 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 460054 T1738 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 460054 T1738 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 460056 T1738 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 460058 T1738 oass.SolrIndexSearcher.<init> Opening Searcher@1b33dde main
[junit4:junit4] 2> 460058 T1738 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/nodes_used_collection_shard1_replica1/data/tlog
[junit4:junit4] 2> 460059 T1738 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 460059 T1738 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 460061 T1764 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@1b33dde main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 460061 T1690 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 460063 T1738 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=down
[junit4:junit4] 2> 460065 T1690 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 460085 T1690 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 460090 T1690 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 460094 T1690 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 460096 T1690 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 460096 T1690 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 460096 T1690 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 460098 T1690 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 460098 T1690 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 460098 T1690 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 460099 T1690 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/nodes_used_collection_shard2_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/nodes_used_collection_shard2_replica2/data/
[junit4:junit4] 2> 460099 T1690 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@8bf924
[junit4:junit4] 2> 460100 T1690 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 460101 T1690 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/nodes_used_collection_shard2_replica2/data
[junit4:junit4] 2> 460101 T1690 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/nodes_used_collection_shard2_replica2/data/index/
[junit4:junit4] 2> 460102 T1690 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/nodes_used_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 460102 T1690 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/nodes_used_collection_shard2_replica2/data/index
[junit4:junit4] 2> 460104 T1690 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@59b19f lockFactory=org.apache.lucene.store.NativeFSLockFactory@12f9c3a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 460105 T1690 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 460107 T1690 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 460107 T1690 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 460108 T1690 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 460109 T1690 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 460109 T1690 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 460110 T1690 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 460110 T1690 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 460111 T1690 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 460111 T1690 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 460115 T1690 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 460118 T1690 oass.SolrIndexSearcher.<init> Opening Searcher@1fa6e14 main
[junit4:junit4] 2> 460118 T1690 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/nodes_used_collection_shard2_replica2/data/tlog
[junit4:junit4] 2> 460119 T1690 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 460119 T1690 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 460121 T1765 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@1fa6e14 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 460122 T1690 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=down
[junit4:junit4] 2> 460646 T1697 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 460647 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49168_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49168/du"}
[junit4:junit4] 2> 460647 T1697 oasc.Overseer$ClusterStateUpdater.createCollection Create collection nodes_used_collection with numShards 2
[junit4:junit4] 2> 460650 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49820_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49820/du"}
[junit4:junit4] 2> 460653 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37077_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37077/du"}
[junit4:junit4] 2> 460655 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:48584_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:48584/du"}
[junit4:junit4] 2> 460658 T1696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 460658 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 460658 T1701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 460658 T1742 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 460658 T1714 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 460658 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 461012 T1750 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica2
[junit4:junit4] 2> 461012 T1750 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:49168/du collection:nodes_used_collection shard:shard1
[junit4:junit4] 2> 461013 T1750 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard1/election
[junit4:junit4] 2> 461019 T1750 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 461021 T1750 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179999
[junit4:junit4] 2> 461050 T1721 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 461050 T1721 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:49820/du collection:nodes_used_collection shard:shard2
[junit4:junit4] 2> 461051 T1721 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4] 2> 461056 T1721 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 461058 T1721 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=180000
[junit4:junit4] 2> 461064 T1738 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 461064 T1738 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:37077/du collection:nodes_used_collection shard:shard1
[junit4:junit4] 2> 461124 T1690 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica2
[junit4:junit4] 2> 461124 T1690 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:48584/du collection:nodes_used_collection shard:shard2
[junit4:junit4] 2> 461522 T1750 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 461522 T1750 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 461522 T1750 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49168/du/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 461523 T1750 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:49168/du START replicas=[http://127.0.0.1:37077/du/nodes_used_collection_shard1_replica1/] nUpdates=100
[junit4:junit4] 2> 461523 T1750 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:49168/du DONE. We have no versions. sync failed.
[junit4:junit4] 2> 461524 T1750 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 461524 T1750 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 461524 T1750 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49168/du/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 461525 T1750 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
[junit4:junit4] 2> ASYNC NEW_CORE C1917 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@54e82f url=http://127.0.0.1:37077/du/nodes_used_collection_shard1_replica1 node=127.0.0.1:37077_du C1917_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=down, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:37077_du, base_url=http://127.0.0.1:37077/du}
[junit4:junit4] 2> 461530 T1736 C1917 P37077 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp=/du path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 461560 T1721 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 461560 T1721 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 461560 T1721 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49820/du/nodes_used_collection_shard2_replica1/
[junit4:junit4] 2> 461561 T1721 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:49820/du START replicas=[http://127.0.0.1:48584/du/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4:junit4] 2> 461561 T1721 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:49820/du DONE. We have no versions. sync failed.
[junit4:junit4] 2> 461561 T1721 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 461562 T1721 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 461562 T1721 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49820/du/nodes_used_collection_shard2_replica1/
[junit4:junit4] 2> 461562 T1721 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard2
[junit4:junit4] 2> ASYNC NEW_CORE C1918 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@1cf3807 url=http://127.0.0.1:48584/du/nodes_used_collection_shard2_replica2 node=127.0.0.1:48584_du C1918_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=down, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:48584_du, base_url=http://127.0.0.1:48584/du}
[junit4:junit4] 2> 461566 T1691 C1918 P48584 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp=/du path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 462168 T1697 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 462178 T1696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 462178 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 462178 T1742 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 462178 T1714 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 462178 T1701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 462179 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 462182 T1750 oasc.ZkController.register We are http://127.0.0.1:49168/du/nodes_used_collection_shard1_replica2/ and leader is http://127.0.0.1:49168/du/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 462182 T1750 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica2 baseURL=http://127.0.0.1:49168/du
[junit4:junit4] 2> 462182 T1750 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 462183 T1750 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=active
[junit4:junit4] 2> 462184 T1750 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 462185 T1750 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=2865
[junit4:junit4] 2> 462218 T1738 oasc.ZkController.register We are http://127.0.0.1:37077/du/nodes_used_collection_shard1_replica1/ and leader is http://127.0.0.1:49168/du/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 462218 T1738 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica1 baseURL=http://127.0.0.1:37077/du
[junit4:junit4] 2> 462219 T1738 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 462219 T1738 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 462218 T1721 oasc.ZkController.register We are http://127.0.0.1:49820/du/nodes_used_collection_shard2_replica1/ and leader is http://127.0.0.1:49820/du/nodes_used_collection_shard2_replica1/
[junit4:junit4] 2> 462219 T1768 C1917 P37077 oasc.RecoveryStrategy.run Starting recovery process. core=nodes_used_collection_shard1_replica1 recoveringAfterStartup=true
[junit4:junit4] 2> 462220 T1721 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica1 baseURL=http://127.0.0.1:49820/du
[junit4:junit4] 2> 462220 T1721 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 462220 T1738 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 462220 T1768 C1917 P37077 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 462220 T1721 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=active
[junit4:junit4] 2> 462221 T1768 C1917 P37077 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=recovering
[junit4:junit4] 2> 462221 T1738 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=2901
[junit4:junit4] 2> 462223 T1768 C1917 P37077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 462223 T1721 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 462223 T1721 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=2904
[junit4:junit4] 2> 462226 T1751 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:37077_du_nodes_used_collection_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 462228 T1690 oasc.ZkController.register We are http://127.0.0.1:48584/du/nodes_used_collection_shard2_replica2/ and leader is http://127.0.0.1:49820/du/nodes_used_collection_shard2_replica1/
[junit4:junit4] 2> 462228 T1690 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica2 baseURL=http://127.0.0.1:48584/du
[junit4:junit4] 2> 462228 T1690 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard2_replica2
[junit4:junit4] 2> 462229 T1690 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 462229 T1769 C1918 P48584 oasc.RecoveryStrategy.run Starting recovery process. core=nodes_used_collection_shard2_replica2 recoveringAfterStartup=true
[junit4:junit4] 2> 462229 T1769 C1918 P48584 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 462230 T1769 C1918 P48584 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=recovering
[junit4:junit4] 2> 462230 T1690 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 462230 T1769 C1918 P48584 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 462231 T1690 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=2912
[junit4:junit4] 2> 462231 T1698 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: nodes_used_collection
[junit4:junit4] 2> 462234 T1724 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:48584_du_nodes_used_collection_shard2_replica2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 462237 T1714 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4] 2> 462238 T1696 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 462238 T1698 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=2865},core=nodes_used_collection_shard1_replica2},null={responseHeader={status=0,QTime=2901},core=nodes_used_collection_shard1_replica1},null={responseHeader={status=0,QTime=2904},core=nodes_used_collection_shard2_replica1},null={responseHeader={status=0,QTime=2912},core=nodes_used_collection_shard2_replica2}}}
[junit4:junit4] 2> 462241 T1708 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2} status=0 QTime=2928
[junit4:junit4] 2> 462242 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 462242 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 462243 T1771 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@106ecba name:ZooKeeperConnection Watcher:127.0.0.1:36211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 462244 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 462245 T1674 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 462247 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 462248 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 462249 T1773 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1540df4 name:ZooKeeperConnection Watcher:127.0.0.1:36211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 462249 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 462250 T1674 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 462254 T1722 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&maxShardsPerNode=2&name=awholynewcollection_0&replicationFactor=3&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 462256 T1696 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 462257 T1698 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "maxShardsPerNode":"2",
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"awholynewcollection_0",
[junit4:junit4] 2> "replicationFactor":"3"}
[junit4:junit4] 2> 462257 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica1 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:37077_du
[junit4:junit4] 2> 462257 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica2 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:45310_du
[junit4:junit4] 2> 462257 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica3 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:49820_du
[junit4:junit4] 2> 462257 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica1 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:48584_du
[junit4:junit4] 2> 462258 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica2 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:49168_du
[junit4:junit4] 2> 462258 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica3 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:37077_du
[junit4:junit4] 2> 462259 T1709 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/awholynewcollection_0_shard1_replica2
[junit4:junit4] 2> 462259 T1709 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 462259 T1736 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard1_replica1
[junit4:junit4] 2> 462260 T1736 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 462259 T1723 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica3' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/awholynewcollection_0_shard1_replica3
[junit4:junit4] 2> 462261 T1723 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 462261 T1735 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica3' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard2_replica3
[junit4:junit4] 2> 462262 T1736 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 462260 T1752 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/awholynewcollection_0_shard2_replica2
[junit4:junit4] 2> 462262 T1752 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 462260 T1692 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/awholynewcollection_0_shard2_replica1
[junit4:junit4] 2> 462263 T1752 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 462260 T1709 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 462263 T1752 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 462263 T1692 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 462262 T1736 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 462262 T1735 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 462262 T1723 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 462263 T1709 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 462265 T1723 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 462265 T1735 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 462266 T1735 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 462266 T1752 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 462265 T1692 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 462266 T1752 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 462266 T1736 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 462266 T1723 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 462267 T1736 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 462267 T1735 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 462267 T1709 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 462267 T1692 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 462269 T1709 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 462268 T1752 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 462268 T1735 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 462268 T1723 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 462270 T1692 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 462269 T1736 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 462271 T1709 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 462271 T1752 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/awholynewcollection_0_shard2_replica2/'
[junit4:junit4] 2> 462272 T1692 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 462272 T1723 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 462272 T1735 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 462273 T1709 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/awholynewcollection_0_shard1_replica2/'
[junit4:junit4] 2> 462273 T1692 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/awholynewcollection_0_shard2_replica1/'
[junit4:junit4] 2> 462273 T1736 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard1_replica1/'
[junit4:junit4] 2> 462274 T1723 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/awholynewcollection_0_shard1_replica3/'
[junit4:junit4] 2> 462275 T1735 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard2_replica3/'
[junit4:junit4] 2> 462329 T1752 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 462330 T1692 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 462331 T1735 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 462345 T1709 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 462355 T1736 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 462373 T1723 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 462413 T1709 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 462414 T1752 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 462415 T1752 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 462417 T1735 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 462418 T1709 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 462427 T1735 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 462428 T1692 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 462429 T1752 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica2] Schema name=test
[junit4:junit4] 2> 462430 T1692 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 462433 T1709 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica2] Schema name=test
[junit4:junit4] 2> 462433 T1735 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica3] Schema name=test
[junit4:junit4] 2> 462436 T1736 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 462436 T1692 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica1] Schema name=test
[junit4:junit4] 2> 462438 T1736 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 462444 T1723 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 462444 T1736 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica1] Schema name=test
[junit4:junit4] 2> 462445 T1723 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 462465 T1723 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica3] Schema name=test
[junit4:junit4] 2> 463318 T1692 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 463330 T1752 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 463332 T1692 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 463334 T1723 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 463335 T1736 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 463336 T1692 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 463339 T1752 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 463343 T1723 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 463344 T1736 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 463345 T1752 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 463347 T1723 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 463350 T1736 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 463354 T1692 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 463365 T1692 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 463368 T1723 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 463369 T1752 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 463370 T1692 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 463372 T1723 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 463372 T1692 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 463373 T1692 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 463373 T1692 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 463374 T1709 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 463375 T1736 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 463376 T1723 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 463377 T1723 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 463377 T1723 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 463377 T1723 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 463373 T1752 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 463378 T1736 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 463375 T1692 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 463378 T1723 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 463380 T1723 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 463379 T1692 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 463380 T1723 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 463380 T1692 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 463381 T1723 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica3] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/awholynewcollection_0_shard1_replica3/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/awholynewcollection_0_shard1_replica3/data/
[junit4:junit4] 2> 463381 T1692 oasc.SolrCore.<init> [awholynewcollection_0_shard2_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/awholynewcollection_0_shard2_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/awholynewcollection_0_shard2_replica1/data/
[junit4:junit4] 2> 463381 T1723 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@8bf924
[junit4:junit4] 2> 463382 T1692 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@8bf924
[junit4:junit4] 2> 463382 T1709 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 463383 T1692 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 463382 T1723 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 463382 T1752 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 463382 T1736 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 463384 T1723 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/awholynewcollection_0_shard1_replica3/data
[junit4:junit4] 2> 463384 T1692 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/awholynewcollection_0_shard2_replica1/data
[junit4:junit4] 2> 463385 T1723 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/awholynewcollection_0_shard1_replica3/data/index/
[junit4:junit4] 2> 463385 T1736 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 463386 T1736 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 463385 T1723 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica3] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/awholynewcollection_0_shard1_replica3/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 463385 T1752 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 463387 T1752 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 463385 T1692 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/awholynewcollection_0_shard2_replica1/data/index/
[junit4:junit4] 2> 463387 T1752 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 463387 T1723 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/awholynewcollection_0_shard1_replica3/data/index
[junit4:junit4] 2> 463386 T1736 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 463386 T1709 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 463388 T1752 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 463389 T1752 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 463387 T1692 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard2_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/awholynewcollection_0_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 463390 T1736 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 463390 T1692 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/awholynewcollection_0_shard2_replica1/data/index
[junit4:junit4] 2> 463390 T1723 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16eb8a8 lockFactory=org.apache.lucene.store.NativeFSLockFactory@14d4d47),segFN=segments_1,generation=1}
[junit4:junit4] 2> 463391 T1723 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 463389 T1752 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 463392 T1752 oasc.SolrCore.<init> [awholynewcollection_0_shard2_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/awholynewcollection_0_shard2_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/awholynewcollection_0_shard2_replica2/data/
[junit4:junit4] 2> 463392 T1692 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@11fb6f1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1ec6459),segFN=segments_1,generation=1}
[junit4:junit4] 2> 463390 T1736 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 463393 T1723 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 463393 T1736 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 463392 T1692 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 463392 T1752 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@8bf924
[junit4:junit4] 2> 463394 T1736 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard1_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard1_replica1/data/
[junit4:junit4] 2> 463393 T1723 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 463396 T1736 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@8bf924
[junit4:junit4] 2> 463395 T1752 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 463397 T1735 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 463397 T1736 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 463396 T1723 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 463396 T1692 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 463399 T1723 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 463398 T1736 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard1_replica1/data
[junit4:junit4] 2> 463398 T1752 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/awholynewcollection_0_shard2_replica2/data
[junit4:junit4] 2> 463400 T1752 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/awholynewcollection_0_shard2_replica2/data/index/
[junit4:junit4] 2> 463400 T1752 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard2_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/awholynewcollection_0_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 463400 T1736 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard1_replica1/data/index/
[junit4:junit4] 2> 463399 T1723 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 463401 T1723 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 463399 T1692 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 463402 T1723 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 463401 T1736 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 463403 T1723 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 463401 T1752 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/awholynewcollection_0_shard2_replica2/data/index
[junit4:junit4] 2> 463401 T1709 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 463403 T1723 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 463403 T1735 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 463403 T1736 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard1_replica1/data/index
[junit4:junit4] 2> 463402 T1692 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 463405 T1752 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1033935 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f46937),segFN=segments_1,generation=1}
[junit4:junit4] 2> 463405 T1692 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 463405 T1752 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 463406 T1692 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 463406 T1692 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 463406 T1736 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1a9e770 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1321518),segFN=segments_1,generation=1}
[junit4:junit4] 2> 463407 T1736 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 463407 T1723 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 463407 T1692 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 463408 T1752 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 463407 T1735 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 463409 T1736 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 463409 T1709 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 463409 T1752 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 463409 T1692 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 463409 T1736 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 463411 T1692 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 463410 T1723 oass.SolrIndexSearcher.<init> Opening Searcher@c0c3c9 main
[junit4:junit4] 2> 463410 T1752 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 463412 T1723 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/awholynewcollection_0_shard1_replica3/data/tlog
[junit4:junit4] 2> 463411 T1736 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 463413 T1723 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 463412 T1752 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 463413 T1723 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 463413 T1736 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 463414 T1752 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 463415 T1752 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 463414 T1692 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 463414 T1736 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 463416 T1736 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 463415 T1752 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 463415 T1709 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 463416 T1752 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 463416 T1736 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 463418 T1709 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 463418 T1709 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 463418 T1723 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica3 state=down
[junit4:junit4] 2> 463418 T1692 oass.SolrIndexSearcher.<init> Opening Searcher@1fad559 main
[junit4:junit4] 2> 463417 T1752 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 463417 T1776 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard1_replica3] Registered new searcher Searcher@c0c3c9 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 463420 T1692 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369292776912/awholynewcollection_0_shard2_replica1/data/tlog
[junit4:junit4] 2> 463419 T1709 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 463418 T1736 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 463421 T1735 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 463421 T1692 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 463421 T1752 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 463422 T1692 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 463421 T1709 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 463421 T1736 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 463423 T1709 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 463424 T1709 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 463424 T1752 oass.SolrIndexSearcher.<init> Opening Searcher@110a53b main
[junit4:junit4] 2> 463424 T1736 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 463424 T1735 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 463424 T1709 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/awholynewcollection_0_shard1_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/awholynewcollection_0_shard1_replica2/data/
[junit4:junit4] 2> 463425 T1752 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/awholynewcollection_0_shard2_replica2/data/tlog
[junit4:junit4] 2> 463426 T1777 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard2_replica1] Registered new searcher Searcher@1fad559 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 463426 T1709 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@8bf924
[junit4:junit4] 2> 463426 T1752 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 463427 T1709 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 463427 T1736 oass.SolrIndexSearcher.<init> Opening Searcher@21dfcd main
[junit4:junit4] 2> 463427 T1752 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 463428 T1709 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/awholynewcollection_0_shard1_replica2/data
[junit4:junit4] 2> 463428 T1735 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 463428 T1692 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica1 state=down
[junit4:junit4] 2> 463428 T1736 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard1_replica1/data/tlog
[junit4:junit4] 2> 463428 T1709 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/awholynewcollection_0_shard1_replica2/data/index/
[junit4:junit4] 2> 463430 T1736 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 463430 T1709 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/awholynewcollection_0_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 463429 T1735 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 463430 T1709 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/awholynewcollection_0_shard1_replica2/data/index
[junit4:junit4] 2> 463430 T1736 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 463431 T1778 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard2_replica2] Registered new searcher Searcher@110a53b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 463430 T1735 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 463440 T1735 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 463440 T1709 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1644f52 lockFactory=org.apache.lucene.store.NativeFSLockFactory@13137e7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 463441 T1709 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 463441 T1752 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica2 state=down
[junit4:junit4] 2> 463441 T1735 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 463442 T1735 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 463442 T1779 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard1_replica1] Registered new searcher Searcher@21dfcd main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 463443 T1735 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 463442 T1736 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica1 state=down
[junit4:junit4] 2> 463443 T1735 oasc.SolrCore.<init> [awholynewcollection_0_shard2_replica3] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard2_replica3/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard2_replica3/data/
[junit4:junit4] 2> 463443 T1709 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 463444 T1735 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@8bf924
[junit4:junit4] 2> 463444 T1709 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 463444 T1735 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 463445 T1709 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 463445 T1735 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard2_replica3/data
[junit4:junit4] 2> 463445 T1709 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 463445 T1735 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard2_replica3/data/index/
[junit4:junit4] 2> 463446 T1709 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 463446 T1709 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 463446 T1735 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard2_replica3] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard2_replica3/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 463447 T1709 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 463447 T1735 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard2_replica3/data/index
[junit4:junit4] 2> 463447 T1709 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 463448 T1709 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 463449 T1735 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@817fc3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6fca47),segFN=segments_1,generation=1}
[junit4:junit4] 2> 463449 T1735 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 463450 T1709 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 463452 T1735 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 463452 T1709 oass.SolrIndexSearcher.<init> Opening Searcher@e99b03 main
[junit4:junit4] 2> 463452 T1735 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 463452 T1709 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369292777109/awholynewcollection_0_shard1_replica2/data/tlog
[junit4:junit4] 2> 463453 T1735 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 463453 T1709 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 463453 T1735 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 463453 T1709 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 463454 T1735 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 463454 T1735 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 463455 T1735 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 463455 T1735 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 463456 T1735 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 463456 T1780 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard1_replica2] Registered new searcher Searcher@e99b03 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 463458 T1709 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica2 state=down
[junit4:junit4] 2> 463458 T1735 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 463460 T1735 oass.SolrIndexSearcher.<init> Opening Searcher@188b1c4 main
[junit4:junit4] 2> 463460 T1735 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_0_shard2_replica3/data/tlog
[junit4:junit4] 2> 463461 T1735 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 463461 T1735 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 463464 T1781 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard2_replica3] Registered new searcher Searcher@188b1c4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 463464 T1735 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica3 state=down
[junit4:junit4] 2> 463687 T1697 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 463688 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49168_du_nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49168_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49168/du"}
[junit4:junit4] 2> 463692 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:49820_du_nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49820_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49820/du"}
[junit4:junit4] 2> 463695 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:37077_du_nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37077_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37077/du"}
[junit4:junit4] 2> 463698 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:48584_du_nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:48584_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:48584/du"}
[junit4:junit4] 2> 463700 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard1_replica3",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49820_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49820/du"}
[junit4:junit4] 2> 463701 T1697 oasc.Overseer$ClusterStateUpdater.createCollection Create collection awholynewcollection_0 with numShards 2
[junit4:junit4] 2> 463704 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard2_replica1",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:48584_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:48584/du"}
[junit4:junit4] 2> 463706 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard2_replica2",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:49168_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:49168/du"}
[junit4:junit4] 2> 463709 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard1_replica1",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37077_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37077/du"}
[junit4:junit4] 2> 463711 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard1_replica2",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:45310_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:45310/du"}
[junit4:junit4] 2> 463714 T1697 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard2_replica3",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:37077_du",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:37077/du"}
[junit4:junit4] 2> 463719 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 463720 T1771 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 463720 T1701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 463720 T1696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 463720 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 463720 T1714 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 463720 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 463720 T1742 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 464228 T1751 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:37077_du_nodes_used_collection_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 464228 T1751 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:37077_du_nodes_used_collection_shard1_replica1&state=recovering&nodeName=127.0.0.1:37077_du&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard1_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2002
[junit4:junit4] 2> 464235 T1724 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:48584_du_nodes_used_collection_shard2_replica2, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 464235 T1724 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:48584_du_nodes_used_collection_shard2_replica2&state=recovering&nodeName=127.0.0.1:48584_du&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard2_replica1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2001
[junit4:junit4] 2> 464420 T1723 oasc.CoreContainer.registerCore registering core: awholynewcollection_0_shard1_replica3
[junit4:junit4] 2> 464420 T1723 oasc.ZkController.register Register replica - core:awholynewcollection_0_shard1_replica3 address:http://127.0.0.1:49820/du collection:awholynewcollection_0 shard:shard1
[junit4:junit4] 2> 464421 T1723 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0/leader_elect/shard1/election
[junit4:junit4] 2> 464428 T1723 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 464430 T1692 oasc.CoreContainer.registerCore registering core: awholynewcollection_0_shard2_replica1
[junit4:junit4] 2> 464430 T1692 oasc.ZkController.register Register replica - core:awholynewcollection_0_shard2_replica1 address:http://127.0.0.1:48584/du collection:awholynewcollection_0 shard:shard2
[junit4:junit4] 2> 464430 T1723 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=3 found=1 timeoutin=180000
[junit4:junit4] 2> 464431 T1692 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0/leader_elect/shard2/election
[junit4:junit4] 2> 464436 T1692 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 464438 T1692 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=3 found=1 timeoutin=179999
[junit4:junit4] 2> 464442 T1752 oasc.CoreContainer.registerCore registering core: awholynewcollection_0_shard2_replica2
[junit4:junit4] 2> 464443 T1752 oasc.ZkController.register Register replica - core:awholynewcollection_0_shard2_replica2 address:http://127.0.0.1:49168/du collection:awholynewcollection_0 shard:shard2
[junit4:junit4] 2> 464444 T1736 oasc.CoreContainer.registerCore registering core: awholynewcollection_0_shard1_replica1
[junit4:junit4] 2> 464444 T1736 oasc.ZkController.register Register replica - core:awholynewcollection_0_shard1_replica1 address:http://127.0.0.1:37077/du collection:awholynewcollection_0 shard:shard1
[junit4:junit4] 2> 464459 T1709 oasc.CoreContainer.registerCore registering core: awholynewcollection_0_shard1_replica2
[junit4:junit4] 2> 464460 T1709 oasc.ZkController.register Register replica - core:awholynewcollection_0_shard1_replica2 address:http://127.0.0.1:45310/du collection:awholynewcollection_0 shard:shard1
[junit4:junit4] 2> 464466 T1735 oasc.CoreContainer.registerCore registering core: awholynewcollection_0_shard2_replica3
[junit4:junit4] 2> 464466 T1735 oasc.ZkController.register Register replica - core:awholynewcollection_0_shard2_replica3 address:http://127.0.0.1:37077/du collection:awholynewcollection_0 shard:shard2
[junit4:junit4] 2> 464931 T1723 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 464932 T1723 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 464932 T1723 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:49820/du/awholynewcollection_0_shard1_replica3/
[junit4:junit4] 2> 464933 T1723 oasu.PeerSync.sync PeerSync: core=awholynewcollection_0_shard1_replica3 url=http://127.0.0.1:49820/du START replicas=[http://127.0.0.1:37077/du/awholynewcollection_0_shard1_replica1/, http://127.0.0.1:45310/du/awholynewcollection_0_shard1_replica2/] nUpdates=100
[junit4:junit4] 2> 464933 T1723 oasu.PeerSync.sync PeerSync: core=awholynewcollection_0_shard1_replica3 url=http://127.0.0.1:49820/du DONE. We have no versions. sync failed.
[junit4:junit4] 2> 464934 T1723 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 464934 T1723 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 464935 T1723 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:49820/du/awholynewcollection_0_shard1_replica3/
[junit4:junit4] 2> 464935 T1723 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0/leaders/shard1
[junit4:junit4] 2> ASYNC NEW_CORE C1919 name=awholynewcollection_0_shard1_replica2 org.apache.solr.core.SolrCore@1cff332 url=http://127.0.0.1:45310/du/awholynewcollection_0_shard1_replica2 node=127.0.0.1:45310_du C1919_STATE=coll:awholynewcollection_0 core:awholynewcollection_0_shard1_replica2 props:{shard=shard1, state=down, core=awholynewcollection_0_shard1_replica2, collection=awholynewcollection_0, node_name=127.0.0.1:45310_du, base_url=http://127.0.0.1:45310/du}
[junit4:junit4] 2> 464939 T1710 C1919 P45310 oasc.SolrCore.execute [awholynewcollection_0_shard1_replica2] webapp=/du path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 464939 T1692 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 464940 T1692 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> ASYNC NEW_CORE C1920 name=awholynewcollection_0_shard1_replica1 org.apache.solr.core.SolrCore@169919c url=http://127.0.0.1:37077/du/awholynewcollection_0_shard1_replica1 node=127.0.0.1:37077_du C1920_STATE=coll:awholynewcollection_0 core:awholynewcollection_0_shard1_replica1 props:{shard=shard1, state=down, core=awholynewcollection_0_shard1_replica1, collection=awholynewcollection_0, node_name=127.0.0.1:37077_du, base_url=http://127.0.0.1:37077/du}
[junit4:junit4] 2> 464940 T1738 C1920 P37077 oasc.SolrCore.execute [awholynewcollection_0_shard1_replica1] webapp=/du path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 464941 T1692 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:48584/du/awholynewcollection_0_shard2_replica1/
[junit4:junit4] 2> 464942 T1692 oasu.PeerSync.sync PeerSync: core=awholynewcollection_0_shard2_replica1 url=http://127.0.0.1:48584/du START replicas=[http://127.0.0.1:49168/du/awholynewcollection_0_shard2_replica2/, http://127.0.0.1:37077/du/awholynewcollection_0_shard2_replica3/] nUpdates=100
[junit4:junit4] 2> 464942 T1692 oasu.PeerSync.sync PeerSync: core=awholynewcollection_0_shard2_replica1 url=http://127.0.0.1:48584/du DONE. We have no versions. sync failed.
[junit4:junit4] 2> 464943 T1692 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 464943 T1692 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 464943 T1692 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:48584/du/awholynewcollection_0_shard2_replica1/
[junit4:junit4] 2> 464944 T1692 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0/leaders/shard2
[junit4:junit4] 2> ASYNC NEW_CORE C1921 name=awholynewcollection_0_shard2_replica2 org.apache.solr.core.SolrCore@fe9269 url=http://127.0.0.1:49168/du/awholynewcollection_0_shard2_replica2 node=127.0.0.1:49168_du C1921_STATE=coll:awholynewcollection_0 core:awholynewcollection_0_shard2_replica2 props:{shard=shard2, state=down, core=awholynewcollection_0_shard2_replica2, collection=awholynewcollection_0, node_name=127.0.0.1:49168_du, base_url=http://127.0.0.1:49168/du}
[junit4:junit4] 2> 464950 T1751 C1921 P49168 oasc.SolrCore.execute [awholynewcollection_0_shard2_replica2] webapp=/du path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=3
[junit4:junit4] 2> ASYNC NEW_CORE C1922 name=awholynewcollection_0_shard2_replica3 org.apache.solr.core.SolrCore@b11ead url=http://127.0.0.1:37077/du/awholynewcollection_0_shard2_replica3 node=127.0.0.1:37077_du C1922_STATE=coll:awholynewcollection_0 core:awholynewcollection_0_shard2_replica3 props:{shard=shard2, state=down, core=awholynewcollection_0_shard2_replica3, collection=awholynewcollection_0, node_name=127.0.0.1:37077_du, base_url=http://127.0.0.1:37077/du}
[junit4:junit4] 2> 464952 T1737 C1922 P37077 oasc.SolrCore.execute [awholynewcollection_0_shard2_replica3] webapp=/du path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 465238 T1697 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 465254 T1773 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 465254 T1728 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 465255 T1771 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 465255 T1742 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 465255 T1701 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 465255 T1714 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 465254 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 465254 T1696 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 465294 T1723 oasc.ZkController.register We are http://127.0.0.1:49820/du/awholynewcollection_0_shard1_replica3/ and leader is http://127.0.0.1:49820/du/awholynewcollection_0_shard1_replica3/
[junit4:junit4] 2> 465294 T1723 oasc.ZkController.register No LogReplay needed for core=awholynewcollection_0_shard1_replica3 baseURL=http://127.0.0.1:49820/du
[junit4:junit4] 2> 465295 T1723 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 465295 T1723 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica3 state=active
[junit4:junit4] 2> 465297 T1723 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 465297 T1723 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=awholynewcollection_0_shard1_replica3&action=CREATE&collection=awholynewcollection_0&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=3038
[junit4:junit4] 2> 465303 T1692 oasc.ZkController.register We are http://127.0.0.1:48584/du/awholynewcollection_0_shard2_replica1/ and leader is http://127.0.0.1:48584/du/awholynewcollection_0_shard2_replica1/
[junit4:junit4] 2> 465304 T1692 oasc.ZkController.register No LogReplay needed for core=awholynewcollection_0_shard2_replica1 baseURL=http://127.0.0.1:48584/du
[junit4:junit4] 2> 465304 T1692 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 465304 T1692 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica1 state=active
[junit4:junit4] 2> 465306 T1692 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 465307 T1692 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=awholynewcollection_0_shard2_replica1&action=CREATE&collection=awholynewcollection_0&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=3047
[junit4:junit4] 2> 465447 T1752 oasc.ZkController.register We are http://127.0.0.1:49168/du/awholynewcollection_0_shard2_replica2/ and leader is http://127.0.0.1:48584/du/awholynewcollection_0_shard2_replica1/
[junit4:junit4] 2> 465448 T1736 oasc.ZkController.register We are http://127.0.0.1:37077/du/awholynewcollection_0_shard1_replica1/ and leader is http://127.0.0.1:49820/du/awholynewcollection_0_shard1_replica3/
[junit4:junit4] 2> 465448 T1752 oasc.ZkController.register No LogReplay needed for core=awholynewcollection_0_shard2_replica2 baseURL=http://127.0.0.1:49168/du
[junit4:junit4] 2> 465449 T1736 oasc.ZkController.register No LogReplay needed for core=awholynewcollection_0_shard1_replica1 baseURL=http://127.0.0.1:37077/du
[junit4:junit4] 2> 465450 T1736 oasc.ZkController.checkRecovery Core needs to recover:awholynewcollection_0_shard1_replica1
[junit4:junit4] 2> 465450 T1736 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 465449 T1752 oasc.ZkController.checkRecovery Core needs to recover:awholynewcollection_0_shard2_replica2
[junit4:junit4] 2> 465452 T1752 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 465451 T1787 C1920 P37077 oasc.RecoveryStrategy.run Starting recovery process. core=awholynewcollection_0_shard1_replica1 recoveringAfterStartup=true
[junit4:junit4] 2> 465452 T1788 C1921 P49168 oasc.RecoveryStrategy.run Starting recovery process. core=awholynewcollection_0_shard2_replica2 recoveringAfterStartup=true
[junit4:junit4] 2> 465453 T1787 C1920 P37077 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 465452 T1736 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 465454 T1787 C1920 P37077 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica1 state=recovering
[junit4:junit4] 2> 465453 T1752 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 465453 T1788 C1921 P49168 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 465456 T1736 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=awholynewcollection_0_shard1_replica1&action=CREATE&collection=awholynewcollection_0&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=3197
[junit4:junit4] 2> 465457 T1787 C1920 P37077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 465456 T1788 C1921 P49168 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica2 state=recovering
[junit4:junit4] 2> 465457 T1752 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=awholynewcollection_0_shard2_replica2&action=CREATE&collection=awholynewcollection_0&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=3197
[junit4:junit4] 2> 465458 T1788 C1921 P49168 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 465461 T1690 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:49168_du_awholynewcollection_0_shard2_replica2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 465461 T1782 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:37077_du_awholynewcollection_0_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 465463 T1709 oasc.ZkController.register We are http://127.0.0.1:45310/du/awholynewcollection_0_shard1_replica2/ and leader is http://127.0.0.1:49820/du/awholynewcollection_0_shard1_replica3/
[junit4:junit4] 2> 465463 T1709 oasc.ZkController.register No LogReplay needed for core=awholynewcollection_0_shard1_replica2 baseURL=http://127.0.0.1:45310/du
[junit4:junit4] 2> 465464 T1709 oasc.ZkController.checkRecovery Core needs to recover:awholynewcollection_0_shard1_replica2
[junit4:junit4] 2> 465464 T1709 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 465464 T1789 C1919 P45310 oasc.RecoveryStrategy.run Starting recovery process. core=awholynewcollection_0_shard1_replica2 recoveringAfterStartup=true
[junit4:junit4] 2> 465464 T1789 C1919 P45310 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 465465 T1709 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 465465 T1789 C1919 P45310 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica2 state=recovering
[junit4:junit4] 2> 465466 T1709 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=awholynewcollection_0_shard1_replica2&action=CREATE&collection=awholynewcollection_0&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=3207
[junit4:junit4] 2> 465466 T1789 C1919 P45310 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 465469 T1723 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:45310_du_awholynewcollection_0_shard1_replica2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 465469 T1735 oasc.ZkController.register We are http://127.0.0.1:37077/du/awholynewcollection_0_shard2_replica3/ and leader is http://127.0.0.1:48584/du/awholynewcollection_0_shard2_replica1/
[junit4:junit4] 2> 465470 T1735 oasc.ZkController.register No LogReplay needed for core=awholynewcollection_0_shard2_replica3 baseURL=http://127.0.0.1:37077/du
[junit4:junit4] 2> 465470 T1735 oasc.ZkController.checkRecovery Core needs to recover:awholynewcollection_0_shard2_replica3
[junit4:junit4] 2> 465470 T1735 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 465471 T1790 C1922 P37077 oasc.RecoveryStrategy.run Starting recovery process. core=awholynewcollection_0_shard2_replica3 recoveringAfterStartup=true
[junit4:junit4] 2> 465471 T1790 C1922 P37077 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 465471 T1735 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 465472 T1790 C1922 P37077 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica3 state=recovering
[junit4:junit4] 2> 465472 T1735 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=awholynewcollection_0_shard2_replica3&action=CREATE&collection=awholynewcollection_0&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=3212
[junit4:junit4] 2> 465473 T1698 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: awholynewcollection_0
[junit4:junit4] 2> 465473 T1790 C1922 P37077 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 465476 T1691 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:37077_du_awholynewcollection_0_shard2_replica3, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 465476 T1728 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000002 state: SyncConnected type NodeDataChanged
[junit4:junit4] 2> 465477 T1696 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 465477 T1698 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000002 complete, response:{success={null={responseHeader={status=0,QTime=3038},core=awholynewcollection_0_shard1_replica3},null={responseHeader={status=0,QTime=3047},core=awholynewcollection_0_shard2_replica1},null={responseHeader={status=0,QTime=3197},core=awholynewcollection_0_shard1_replica1},null={responseHeader={status=0,QTime=3197},core=awholynewcollection_0_shard2_replica2},null={responseHeader={status=0,QTime=3207},core=awholynewcollection_0_shard1_replica2},null={responseHeader={status=0,QTime=3212},core=awholynewcollection_0_shard2_replica3}}}
[junit4:junit4] 2> 465479 T1722 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=2&maxShardsPerNode=2&name=awholynewcollection_0&replicationFactor=3&action=CREATE&wt=javabin&version=2} status=0 QTime=3225
[junit4:junit4] 2> 465480 T1674 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 465482 T1674 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 465483 T1792 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@106aa58 name:ZooKeeperConnection Watcher:127.0.0.1:36211/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 465483 T1674 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 465484 T1674 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 465489 T1722 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=1&maxShardsPerNode=1&name=awholynewcollection_1&replicationFactor=3&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 465490 T1696 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 465491 T1698 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000004 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "maxShardsPerNode":"1",
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"awholynewcollection_1",
[junit4:junit4] 2> "replicationFactor":"3"}
[junit4:junit4] 2> 465491 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard1_replica1 as part of slice shard1 of collection awholynewcollection_1 on 127.0.0.1:37077_du
[junit4:junit4] 2> 465492 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard1_replica2 as part of slice shard1 of collection awholynewcollection_1 on 127.0.0.1:49820_du
[junit4:junit4] 2> 465492 T1698 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_1_shard1_replica3 as part of slice shard1 of collection awholynewcollection_1 on 127.0.0.1:49168_du
[junit4:junit4] 2> 465493 T1738 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard1_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_1_shard1_replica1
[junit4:junit4] 2> 465493 T1751 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard1_replica3' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/awholynewcollection_1_shard1_replica3
[junit4:junit4] 2> 465493 T1724 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_1_shard1_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/awholynewcollection_1_shard1_replica2
[junit4:junit4] 2> 465494 T1724 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 2> 465494 T1751 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 2> 465494 T1738 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 2> 465495 T1724 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4] 2> 465495 T1738 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4] 2> 465495 T1724 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 465495 T1751 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4] 2> 465496 T1738 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 465496 T1751 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 465496 T1724 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 465497 T1724 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_1
[junit4:junit4] 2> 465497 T1738 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 465497 T1751 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 465498 T1738 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_1
[junit4:junit4] 2> 465498 T1751 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_1
[junit4:junit4] 2> 465499 T1724 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 2> 465500 T1738 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 2> 465500 T1751 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 2> 465501 T1724 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369292778274/awholynewcollection_1_shard1_replica2/'
[junit4:junit4] 2> 465501 T1738 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369292779428/awholynewcollection_1_shard1_replica1/'
[junit4:junit4] 2> 465501 T1751 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369292780609/awholynewcollection_1_shard1_replica3/'
[junit4:junit4] 2> 465544 T1738 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 465547 T1724 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 465555 T1751 oasc.SolrConfig.<init> Using Lucene MatchVe
[...truncated too long message...]
junit4:junit4] 1> {
[junit4:junit4] 1> "core":"awholynewcollection_2_shard3_replica1",
[junit4:junit4] 1> "node_name":"127.0.0.1:48584_du",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:48584/du"}
[junit4:junit4] 1> /solr/collections/control_collection (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/awholynewcollection_1 (2)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "configName":"conf1",
[junit4:junit4] 1> "router":"compositeId"}
[junit4:junit4] 1> /solr/collections/awholynewcollection_1/leader_elect (1)
[junit4:junit4] 1> /solr/collections/awholynewcollection_1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/awholynewcollection_1/leader_elect/shard1/election (3)
[junit4:junit4] 1> /solr/collections/awholynewcollection_1/leader_elect/shard1/election/89737971380649996-127.0.0.1:49168_du_awholynewcollection_1_shard1_replica3-n_0000000002 (0)
[junit4:junit4] 1> /solr/collections/awholynewcollection_1/leader_elect/shard1/election/89737971380649992-127.0.0.1:49820_du_awholynewcollection_1_shard1_replica2-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/awholynewcollection_1/leader_elect/shard1/election/89737971380649994-127.0.0.1:37077_du_awholynewcollection_1_shard1_replica1-n_0000000001 (0)
[junit4:junit4] 1> /solr/collections/awholynewcollection_1/leaders (1)
[junit4:junit4] 1> /solr/collections/awholynewcollection_1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "core":"awholynewcollection_1_shard1_replica2",
[junit4:junit4] 1> "node_name":"127.0.0.1:49820_du",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:49820/du"}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (5)
[junit4:junit4] 1> /solr/overseer_elect/election/89737971380649987-127.0.0.1:48584_du-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89737971380649996-127.0.0.1:49168_du-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89737971380649994-127.0.0.1:37077_du-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89737971380649990-127.0.0.1:45310_du-n_0000000001 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89737971380649992-127.0.0.1:49820_du-n_0000000002 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89737971380649987-127.0.0.1:48584_du-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=437E8AC0DD05609D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fr_CA -Dtests.timezone=America/North_Dakota/New_Salem -Dtests.file.encoding=ISO-8859-1
[junit4:junit4] FAILURE 273s J1 | CollectionsAPIDistributedZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([437E8AC0DD05609D:C29804D8AA5A00A1]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
[junit4:junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
[junit4:junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
[junit4:junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
[junit4:junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 726907 T1674 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 273190 T1673 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 3 leaked thread(s).
[junit4:junit4] 2> 728113 T1756 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewcollection_3/leader_elect/shard2/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 728113 T1756 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 728113 T1756 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 728114 T1756 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=494), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=494)), sim=DefaultSimilarity, locale=fr_CA, timezone=America/North_Dakota/New_Salem
[junit4:junit4] 2> NOTE: Linux 3.2.0-41-generic i386/Oracle Corporation 1.7.0_21 (32-bit)/cpus=8,threads=1,free=62298472,total=182263808
[junit4:junit4] 2> NOTE: All tests run in this JVM: [ConvertedLegacyTest, ZkSolrClientTest, TestWordDelimiterFilterFactory, DocValuesMultiTest, TestSolrQueryParserResource, DisMaxRequestHandlerTest, FieldAnalysisRequestHandlerTest, ShardRoutingCustomTest, TestCollationField, RequiredFieldsTest, TestComponentsName, TestBinaryResponseWriter, TestDistributedSearch, HardAutoCommitTest, TestAddFieldRealTimeGet, TestUtils, TestRandomFaceting, TestCoreDiscovery, TestSuggestSpellingConverter, UpdateParamsTest, PingRequestHandlerTest, TestLazyCores, TestCodecSupport, TestElisionMultitermQuery, TestRecovery, DefaultValueUpdateProcessorTest, PathHierarchyTokenizerFactoryTest, TestAnalyzedSuggestions, TestIndexingPerformance, HighlighterConfigTest, IndexReaderFactoryTest, SolrIndexConfigTest, TestNumberUtils, SolrPluginUtilsTest, TestStressLucene, TestShardHandlerFactory, DistanceFunctionTest, SolrTestCaseJ4Test, TestStressReorder, TestMultiCoreConfBootstrap, TestCSVResponseWriter, SolrRequestParserTest, CopyFieldTest, TestLuceneMatchVersion, SignatureUpdateProcessorFactoryTest, TestRTGBase, SpellPossibilityIteratorTest, IndexSchemaRuntimeFieldTest, TestSolr4Spatial, SimplePostToolTest, OpenExchangeRatesOrgProviderTest, BadIndexSchemaTest, BasicDistributedZk2Test, DirectSolrSpellCheckerTest, TestPostingsSolrHighlighter, TestFastOutputStream, NumericFieldsTest, TestHashPartitioner, TestFieldTypeResource, SpellCheckCollatorTest, TestCopyFieldCollectionResource, TestDistributedGrouping, OutputWriterTest, URLClassifyProcessorTest, ZkNodePropsTest, JSONWriterTest, IndexBasedSpellCheckerTest, TestDFRSimilarityFactory, DocumentAnalysisRequestHandlerTest, ChaosMonkeySafeLeaderTest, CollectionsAPIDistributedZkTest]
[junit4:junit4] Completed on J1 in 274.47s, 1 test, 1 failure <<< FAILURES!
[...truncated 456 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:380: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:360: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:181: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:437: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1240: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:884: There were test failures: 295 suites, 1231 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 38 minutes 29 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_21 -server -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure