You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/05/29 21:45:46 UTC
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 519 -
Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/519/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
There are still nodes recoverying - waited for 230 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
at __randomizedtesting.SeedInfo.seed([3A2C8C600B7111A:824446DE77E87126]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9566 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4:junit4] 2> 2114281 T4551 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 2114289 T4551 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-CollectionsAPIDistributedZkTest-1369855061821
[junit4:junit4] 2> 2114292 T4551 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 2114294 T4552 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 2114399 T4551 oasc.ZkTestServer.run start zk server on port:50532
[junit4:junit4] 2> 2114406 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2114413 T4558 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6c28d496 name:ZooKeeperConnection Watcher:127.0.0.1:50532 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2114414 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2114414 T4551 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 2114426 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2114431 T4560 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5837c5f4 name:ZooKeeperConnection Watcher:127.0.0.1:50532/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2114432 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2114432 T4551 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 2114440 T4551 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 2114446 T4551 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 2114457 T4551 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 2114467 T4551 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 2114468 T4551 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 2114481 T4551 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 2114482 T4551 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 2114489 T4551 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 2114490 T4551 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 2114496 T4551 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 2114497 T4551 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 2114509 T4551 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 2114510 T4551 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 2114519 T4551 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 2114521 T4551 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 2114528 T4551 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 2114529 T4551 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 2114537 T4551 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 2114538 T4551 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 2114546 T4551 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 2114546 T4551 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 2115169 T4551 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2115178 T4551 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50535
[junit4:junit4] 2> 2115179 T4551 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2115180 T4551 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2115180 T4551 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087
[junit4:junit4] 2> 2115181 T4551 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/solr.xml
[junit4:junit4] 2> 2115181 T4551 oasc.CoreContainer.<init> New CoreContainer 711224172
[junit4:junit4] 2> 2115182 T4551 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/'
[junit4:junit4] 2> 2115183 T4551 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/'
[junit4:junit4] 2> 2115364 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087
[junit4:junit4] 2> 2115366 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/collection1
[junit4:junit4] 2> 2115367 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/collection1/conf
[junit4:junit4] 2> 2115556 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/collection1/conf/xslt
[junit4:junit4] 2> 2115561 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/collection1/lib
[junit4:junit4] 2> 2115563 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/collection1/lib/classes
[junit4:junit4] 2> 2115566 T4551 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/conf/core.properties, adding to cores
[junit4:junit4] 2> 2115584 T4551 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 2115585 T4551 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2115585 T4551 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 2115586 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2115587 T4551 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2115587 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2115588 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2115588 T4551 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2115589 T4551 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2115590 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 2115623 T4551 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2115624 T4551 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50532/solr
[junit4:junit4] 2> 2115625 T4551 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2115628 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2115633 T4571 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@9b2fa6a name:ZooKeeperConnection Watcher:127.0.0.1:50532 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2115634 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2115638 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2115662 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2115668 T4573 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@59c7d91f name:ZooKeeperConnection Watcher:127.0.0.1:50532/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2115668 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2115672 T4551 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 2115689 T4551 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 2115698 T4551 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 2115703 T4551 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50535_
[junit4:junit4] 2> 2115706 T4551 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50535_
[junit4:junit4] 2> 2115714 T4551 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 2115729 T4551 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 2115739 T4551 oasc.Overseer.start Overseer (id=89774821332615171-127.0.0.1:50535_-n_0000000000) starting
[junit4:junit4] 2> 2115753 T4551 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 2115767 T4575 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 2115770 T4551 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 2115777 T4551 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 2115782 T4551 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2115793 T4551 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2115793 T4551 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2115794 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2115801 T4574 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 2115823 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 2115826 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2115833 T4578 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@37c4e561 name:ZooKeeperConnection Watcher:127.0.0.1:50532/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2115834 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2115836 T4551 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2115843 T4551 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 2116398 T4551 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2116407 T4551 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50539
[junit4:junit4] 2> 2116410 T4551 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2116414 T4551 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2116416 T4551 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374
[junit4:junit4] 2> 2116417 T4551 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/solr.xml
[junit4:junit4] 2> 2116417 T4551 oasc.CoreContainer.<init> New CoreContainer 77295191
[junit4:junit4] 2> 2116418 T4551 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/'
[junit4:junit4] 2> 2116419 T4551 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/'
[junit4:junit4] 2> 2116618 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374
[junit4:junit4] 2> 2116621 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/collection1
[junit4:junit4] 2> 2116622 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/collection1/conf
[junit4:junit4] 2> 2116821 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/collection1/conf/xslt
[junit4:junit4] 2> 2116825 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/collection1/lib
[junit4:junit4] 2> 2116827 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/collection1/lib/classes
[junit4:junit4] 2> 2116830 T4551 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/conf/core.properties, adding to cores
[junit4:junit4] 2> 2116846 T4551 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 2116848 T4551 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2116849 T4551 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 2116849 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2116850 T4551 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2116850 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2116851 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2116851 T4551 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2116852 T4551 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2116853 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 2116886 T4551 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2116887 T4551 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50532/solr
[junit4:junit4] 2> 2116888 T4551 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2116890 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2116894 T4589 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35b9eabf name:ZooKeeperConnection Watcher:127.0.0.1:50532 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2116895 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2116900 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2116922 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2116928 T4591 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19c97146 name:ZooKeeperConnection Watcher:127.0.0.1:50532/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2116928 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2116939 T4551 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2117948 T4551 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50539_
[junit4:junit4] 2> 2117951 T4551 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50539_
[junit4:junit4] 2> 2117961 T4578 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2117962 T4573 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2117970 T4591 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 2117975 T4551 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2117976 T4551 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2117976 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2118580 T4551 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2118590 T4551 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50542
[junit4:junit4] 2> 2118592 T4551 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2118592 T4551 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2118593 T4551 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535
[junit4:junit4] 2> 2118595 T4551 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/solr.xml
[junit4:junit4] 2> 2118596 T4551 oasc.CoreContainer.<init> New CoreContainer 1067444561
[junit4:junit4] 2> 2118597 T4551 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/'
[junit4:junit4] 2> 2118597 T4551 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/'
[junit4:junit4] 2> 2118768 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535
[junit4:junit4] 2> 2118770 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/collection1
[junit4:junit4] 2> 2118772 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/collection1/conf
[junit4:junit4] 2> 2118967 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/collection1/conf/xslt
[junit4:junit4] 2> 2118972 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/collection1/lib
[junit4:junit4] 2> 2118973 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/collection1/lib/classes
[junit4:junit4] 2> 2118976 T4551 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/conf/core.properties, adding to cores
[junit4:junit4] 2> 2119000 T4551 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 2119000 T4551 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2119001 T4551 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 2119002 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2119002 T4551 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2119003 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2119003 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2119004 T4551 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2119004 T4551 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2119005 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 2119055 T4551 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2119056 T4551 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50532/solr
[junit4:junit4] 2> 2119057 T4551 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2119060 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2119069 T4603 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7c5f506c name:ZooKeeperConnection Watcher:127.0.0.1:50532 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2119074 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2119079 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2119102 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2119107 T4605 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@35525190 name:ZooKeeperConnection Watcher:127.0.0.1:50532/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2119108 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2119118 T4551 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2120129 T4551 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50542_
[junit4:junit4] 2> 2120140 T4551 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50542_
[junit4:junit4] 2> 2120156 T4605 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2120161 T4578 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2120161 T4591 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2120162 T4573 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 2120174 T4551 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2120176 T4551 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2120181 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2120801 T4551 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2120810 T4551 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50545
[junit4:junit4] 2> 2120811 T4551 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2120811 T4551 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2120812 T4551 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734
[junit4:junit4] 2> 2120813 T4551 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/solr.xml
[junit4:junit4] 2> 2120813 T4551 oasc.CoreContainer.<init> New CoreContainer 1386808458
[junit4:junit4] 2> 2120814 T4551 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/'
[junit4:junit4] 2> 2120815 T4551 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/'
[junit4:junit4] 2> 2120982 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734
[junit4:junit4] 2> 2120984 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/collection1
[junit4:junit4] 2> 2120986 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/collection1/conf
[junit4:junit4] 2> 2121167 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/collection1/conf/xslt
[junit4:junit4] 2> 2121173 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/collection1/lib
[junit4:junit4] 2> 2121175 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/collection1/lib/classes
[junit4:junit4] 2> 2121178 T4551 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/conf/core.properties, adding to cores
[junit4:junit4] 2> 2121193 T4551 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 2121194 T4551 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2121194 T4551 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 2121195 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2121195 T4551 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2121196 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2121197 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2121197 T4551 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2121198 T4551 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2121198 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 2121227 T4551 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2121234 T4551 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50532/solr
[junit4:junit4] 2> 2121235 T4551 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2121237 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2121242 T4617 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@429d0062 name:ZooKeeperConnection Watcher:127.0.0.1:50532 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2121243 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2121248 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2121272 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2121278 T4619 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@49b43b name:ZooKeeperConnection Watcher:127.0.0.1:50532/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2121278 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2121291 T4551 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2122301 T4551 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50545_
[junit4:junit4] 2> 2122304 T4551 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50545_
[junit4:junit4] 2> 2122312 T4619 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2122312 T4605 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2122328 T4551 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2122329 T4551 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2122330 T4578 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2122330 T4573 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2122330 T4591 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 2122330 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2122989 T4551 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 2123014 T4551 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50548
[junit4:junit4] 2> 2123015 T4551 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 2123015 T4551 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 2123016 T4551 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884
[junit4:junit4] 2> 2123016 T4551 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/solr.xml
[junit4:junit4] 2> 2123016 T4551 oasc.CoreContainer.<init> New CoreContainer 1694485206
[junit4:junit4] 2> 2123017 T4551 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/'
[junit4:junit4] 2> 2123018 T4551 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/'
[junit4:junit4] 2> 2123237 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884
[junit4:junit4] 2> 2123247 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/collection1
[junit4:junit4] 2> 2123250 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/collection1/conf
[junit4:junit4] 2> 2123488 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/collection1/conf/xslt
[junit4:junit4] 2> 2123493 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/collection1/lib
[junit4:junit4] 2> 2123495 T4551 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/collection1/lib/classes
[junit4:junit4] 2> 2123501 T4551 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/conf/core.properties, adding to cores
[junit4:junit4] 2> 2123530 T4551 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 2123533 T4551 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 2123534 T4551 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 2123535 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 2123540 T4551 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 2123541 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 2123541 T4551 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 2123542 T4551 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 2123543 T4551 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 2123544 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 2123580 T4551 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 2123581 T4551 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50532/solr
[junit4:junit4] 2> 2123581 T4551 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 2123583 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2123601 T4631 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@630bd64d name:ZooKeeperConnection Watcher:127.0.0.1:50532 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2123602 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2123616 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 2123656 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2123662 T4633 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@23def9de name:ZooKeeperConnection Watcher:127.0.0.1:50532/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2123663 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2123695 T4551 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2124723 T4551 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50548_
[junit4:junit4] 2> 2124728 T4551 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50548_
[junit4:junit4] 2> 2124737 T4573 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2124738 T4591 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2124738 T4633 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2124740 T4619 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2124740 T4578 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2124741 T4605 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 2124760 T4551 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 2124760 T4551 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 2124761 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2124790 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2124821 T4584 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 2124829 T4573 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 2124836 T4575 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "maxShardsPerNode":null,
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"nodes_used_collection",
[junit4:junit4] 2> "replicationFactor":"2"}
[junit4:junit4] 2> 2124836 T4575 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:50535_
[junit4:junit4] 2> 2124837 T4575 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:50548_
[junit4:junit4] 2> 2124838 T4575 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:50539_
[junit4:junit4] 2> 2124851 T4575 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:50545_
[junit4:junit4] 2> 2124866 T4628 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/nodes_used_collection_shard1_replica2
[junit4:junit4] 2> 2124866 T4569 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2124867 T4569 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 2124866 T4586 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2124868 T4586 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 2124866 T4628 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 2124869 T4613 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/nodes_used_collection_shard2_replica2
[junit4:junit4] 2> 2124870 T4613 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 2124871 T4569 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 2124873 T4569 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2124875 T4586 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 2124875 T4628 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 2124876 T4586 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2124876 T4628 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2124879 T4613 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 2124879 T4613 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2124884 T4569 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2124884 T4569 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 2124889 T4586 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2124890 T4586 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 2124891 T4613 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2124892 T4613 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 2124893 T4628 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2124893 T4628 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 2124900 T4569 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 2124902 T4586 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 2124908 T4613 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 2124910 T4628 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 2124915 T4569 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/'
[junit4:junit4] 2> 2124915 T4586 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/'
[junit4:junit4] 2> 2124924 T4613 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/nodes_used_collection_shard2_replica2/'
[junit4:junit4] 2> 2124941 T4628 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/nodes_used_collection_shard1_replica2/'
[junit4:junit4] 2> 2125177 T4569 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2125200 T4613 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2125214 T4628 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2125270 T4586 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2125472 T4569 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2125484 T4628 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2125486 T4613 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2125495 T4569 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2125504 T4628 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2125505 T4569 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica1] Schema name=test
[junit4:junit4] 2> 2125512 T4613 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2125519 T4613 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica2] Schema name=test
[junit4:junit4] 2> 2125522 T4628 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica2] Schema name=test
[junit4:junit4] 2> 2125575 T4586 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2125627 T4586 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2125634 T4586 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica1] Schema name=test
[junit4:junit4] 2> 2128454 T4613 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2128499 T4613 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2128506 T4613 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2128595 T4586 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2128612 T4586 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2128616 T4586 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2128636 T4628 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2128672 T4628 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2128674 T4613 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2128679 T4628 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2128695 T4569 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2128699 T4613 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2128714 T4613 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2128724 T4613 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2128725 T4613 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2128725 T4613 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2128727 T4569 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2128734 T4569 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2128745 T4613 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2128746 T4613 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2128746 T4613 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2128748 T4613 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/nodes_used_collection_shard2_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/nodes_used_collection_shard2_replica2/data/
[junit4:junit4] 2> 2128753 T4613 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@23441f2b
[junit4:junit4] 2> 2128759 T4613 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2128770 T4613 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/nodes_used_collection_shard2_replica2/data
[junit4:junit4] 2> 2128776 T4613 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/nodes_used_collection_shard2_replica2/data/index/
[junit4:junit4] 2> 2128777 T4613 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/nodes_used_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2128774 T4586 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2128779 T4613 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/nodes_used_collection_shard2_replica2/data/index
[junit4:junit4] 2> 2128797 T4613 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c13b0c1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2ae56f0c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2128797 T4613 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2128799 T4586 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2128804 T4613 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2128805 T4613 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2128806 T4613 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2128807 T4613 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2128811 T4613 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2128812 T4613 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2128816 T4613 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2128817 T4613 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2128818 T4613 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2128835 T4628 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2128850 T4613 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2128856 T4628 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2128858 T4586 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2128864 T4613 oass.SolrIndexSearcher.<init> Opening Searcher@2c00d4cd main
[junit4:junit4] 2> 2128867 T4613 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/nodes_used_collection_shard2_replica2/data/tlog
[junit4:junit4] 2> 2128868 T4613 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2128869 T4613 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2128871 T4628 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2128881 T4586 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2128884 T4586 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2128884 T4586 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2128887 T4640 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@2c00d4cd main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2128888 T4613 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=down
[junit4:junit4] 2> 2128883 T4628 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2128890 T4628 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2128890 T4628 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2128896 T4569 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2128903 T4586 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2128904 T4586 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2128904 T4586 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2128905 T4586 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/data/
[junit4:junit4] 2> 2128906 T4586 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@23441f2b
[junit4:junit4] 2> 2128907 T4586 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2128909 T4628 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2128910 T4628 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2128910 T4628 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2128911 T4628 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/nodes_used_collection_shard1_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/nodes_used_collection_shard1_replica2/data/
[junit4:junit4] 2> 2128911 T4628 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@23441f2b
[junit4:junit4] 2> 2128912 T4628 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2128915 T4586 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/data
[junit4:junit4] 2> 2128916 T4586 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/data/index/
[junit4:junit4] 2> 2128916 T4586 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2128917 T4586 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/data/index
[junit4:junit4] 2> 2128920 T4569 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2128924 T4628 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/nodes_used_collection_shard1_replica2/data
[junit4:junit4] 2> 2128924 T4628 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/nodes_used_collection_shard1_replica2/data/index/
[junit4:junit4] 2> 2128925 T4628 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/nodes_used_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2128926 T4628 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/nodes_used_collection_shard1_replica2/data/index
[junit4:junit4] 2> 2128942 T4586 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c38e69e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5772d05a),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2128943 T4586 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2128948 T4586 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2128949 T4586 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2128950 T4586 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2128952 T4586 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2128953 T4586 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2128953 T4586 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2128954 T4586 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2128955 T4586 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2128956 T4586 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2128961 T4628 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16581dd7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a50e5c7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2128961 T4628 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2128974 T4628 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2128974 T4628 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2128977 T4628 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2128986 T4628 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2128987 T4628 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2128988 T4628 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2128989 T4628 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2128990 T4628 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2128991 T4628 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2129002 T4586 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2129002 T4628 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2129008 T4569 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2129019 T4628 oass.SolrIndexSearcher.<init> Opening Searcher@56a09036 main
[junit4:junit4] 2> 2129021 T4628 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/nodes_used_collection_shard1_replica2/data/tlog
[junit4:junit4] 2> 2129022 T4569 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2129023 T4569 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2129023 T4628 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2129025 T4628 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2129021 T4586 oass.SolrIndexSearcher.<init> Opening Searcher@447a6953 main
[junit4:junit4] 2> 2129045 T4586 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/data/tlog
[junit4:junit4] 2> 2129035 T4628 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=down
[junit4:junit4] 2> 2129030 T4641 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@56a09036 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2129023 T4569 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2129047 T4586 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2129054 T4586 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2129060 T4642 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@447a6953 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2129064 T4586 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=down
[junit4:junit4] 2> 2129068 T4569 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2129068 T4569 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2129069 T4569 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2129070 T4569 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/data/
[junit4:junit4] 2> 2129072 T4569 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@23441f2b
[junit4:junit4] 2> 2129076 T4569 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2129084 T4569 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/data
[junit4:junit4] 2> 2129084 T4569 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/data/index/
[junit4:junit4] 2> 2129084 T4569 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2129085 T4569 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/data/index
[junit4:junit4] 2> 2129097 T4569 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@714cf3aa lockFactory=org.apache.lucene.store.NativeFSLockFactory@2130ede),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2129098 T4569 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2129128 T4569 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2129130 T4569 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2129131 T4569 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2129132 T4569 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2129133 T4569 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2129134 T4569 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2129134 T4569 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2129135 T4569 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2129136 T4569 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2129167 T4569 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2129192 T4569 oass.SolrIndexSearcher.<init> Opening Searcher@1e9a2f49 main
[junit4:junit4] 2> 2129194 T4569 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/data/tlog
[junit4:junit4] 2> 2129196 T4569 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2129197 T4569 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2129204 T4643 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@1e9a2f49 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2129208 T4569 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=down
[junit4:junit4] 2> 2129425 T4574 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2129428 T4574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50545_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50545"}
[junit4:junit4] 2> 2129428 T4574 oasc.Overseer$ClusterStateUpdater.createCollection Create collection nodes_used_collection with numShards 2
[junit4:junit4] 2> 2129441 T4574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50548_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50548"}
[junit4:junit4] 2> 2129453 T4574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50539_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50539"}
[junit4:junit4] 2> 2129468 T4574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50535_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50535"}
[junit4:junit4] 2> 2129502 T4591 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2129503 T4605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2129503 T4619 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2129503 T4633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2129503 T4578 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2129505 T4573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2129903 T4613 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica2
[junit4:junit4] 2> 2129904 T4613 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:50545 collection:nodes_used_collection shard:shard2
[junit4:junit4] 2> 2129906 T4613 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4] 2> 2129953 T4613 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2129963 T4613 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179998
[junit4:junit4] 2> 2130062 T4628 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica2
[junit4:junit4] 2> 2130062 T4628 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:50548 collection:nodes_used_collection shard:shard1
[junit4:junit4] 2> 2130065 T4628 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard1/election
[junit4:junit4] 2> 2130070 T4586 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2130070 T4586 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:50539 collection:nodes_used_collection shard:shard2
[junit4:junit4] 2> 2130097 T4628 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 2130124 T4628 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179997
[junit4:junit4] 2> 2130215 T4569 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2130216 T4569 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:50535 collection:nodes_used_collection shard:shard1
[junit4:junit4] 2> 2130468 T4613 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2130469 T4613 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2130470 T4613 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50545/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 2130471 T4613 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:50545 START replicas=[http://127.0.0.1:50539/nodes_used_collection_shard2_replica1/] nUpdates=100
[junit4:junit4] 2> 2130472 T4613 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:50545 DONE. We have no versions. sync failed.
[junit4:junit4] 2> 2130472 T4613 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 2130473 T4613 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 2130473 T4613 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50545/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 2130474 T4613 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard2
[junit4:junit4] 2> ASYNC NEW_CORE C3395 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@78ea44c url=http://127.0.0.1:50539/nodes_used_collection_shard2_replica1 node=127.0.0.1:50539_ C3395_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=down, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50539_, base_url=http://127.0.0.1:50539}
[junit4:junit4] 2> 2130519 T4585 C3395 P50539 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=4
[junit4:junit4] 2> 2130634 T4628 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 2130634 T4628 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 2130635 T4628 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50548/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 2130636 T4628 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:50548 START replicas=[http://127.0.0.1:50535/nodes_used_collection_shard1_replica1/] nUpdates=100
[junit4:junit4] 2> 2130637 T4628 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:50548 DONE. We have no versions. sync failed.
[junit4:junit4] 2> 2130637 T4628 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 2130638 T4628 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 2130638 T4628 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50548/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 2130639 T4628 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
[junit4:junit4] 2> ASYNC NEW_CORE C3396 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@2b2e9560 url=http://127.0.0.1:50535/nodes_used_collection_shard1_replica1 node=127.0.0.1:50535_ C3396_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=down, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50535_, base_url=http://127.0.0.1:50535}
[junit4:junit4] 2> 2130660 T4566 C3396 P50535 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 2131087 T4574 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2131127 T4578 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2131127 T4591 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2131127 T4619 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2131127 T4573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2131128 T4605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2131127 T4633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2131141 T4586 oasc.ZkController.register We are http://127.0.0.1:50539/nodes_used_collection_shard2_replica1/ and leader is http://127.0.0.1:50545/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 2131141 T4586 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica1 baseURL=http://127.0.0.1:50539
[junit4:junit4] 2> 2131142 T4586 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2131142 T4586 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 2131144 T4646 C3395 P50539 oasc.RecoveryStrategy.run Starting recovery process. core=nodes_used_collection_shard2_replica1 recoveringAfterStartup=true
[junit4:junit4] 2> 2131145 T4586 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2131146 T4646 C3395 P50539 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 2131147 T4646 C3395 P50539 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=recovering
[junit4:junit4] 2> 2131148 T4586 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=6282
[junit4:junit4] 2> 2131150 T4646 C3395 P50539 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2131151 T4613 oasc.ZkController.register We are http://127.0.0.1:50545/nodes_used_collection_shard2_replica2/ and leader is http://127.0.0.1:50545/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 2131151 T4613 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica2 baseURL=http://127.0.0.1:50545
[junit4:junit4] 2> 2131152 T4613 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 2131152 T4613 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=active
[junit4:junit4] 2> 2131163 T4613 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2131166 T4613 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=6297
[junit4:junit4] 2> 2131174 T4628 oasc.ZkController.register We are http://127.0.0.1:50548/nodes_used_collection_shard1_replica2/ and leader is http://127.0.0.1:50548/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 2131176 T4628 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica2 baseURL=http://127.0.0.1:50548
[junit4:junit4] 2> 2131178 T4628 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 2131179 T4628 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=active
[junit4:junit4] 2> 2131183 T4628 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2131185 T4628 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=6319
[junit4:junit4] 2> 2131195 T4614 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:50539__nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 2131243 T4569 oasc.ZkController.register We are http://127.0.0.1:50535/nodes_used_collection_shard1_replica1/ and leader is http://127.0.0.1:50548/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 2131244 T4569 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica1 baseURL=http://127.0.0.1:50535
[junit4:junit4] 2> 2131244 T4569 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2131245 T4569 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 2131246 T4647 C3396 P50535 oasc.RecoveryStrategy.run Starting recovery process. core=nodes_used_collection_shard1_replica1 recoveringAfterStartup=true
[junit4:junit4] 2> 2131248 T4569 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2131248 T4647 C3396 P50535 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 2131249 T4647 C3396 P50535 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=recovering
[junit4:junit4] 2> 2131250 T4569 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=6384
[junit4:junit4] 2> 2131251 T4575 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: nodes_used_collection
[junit4:junit4] 2> 2131251 T4647 C3396 P50535 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 2131277 T4591 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4] 2> 2131277 T4626 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:50535__nodes_used_collection_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 2131282 T4573 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 2131282 T4575 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=6282},core=nodes_used_collection_shard2_replica1},null={responseHeader={status=0,QTime=6297},core=nodes_used_collection_shard2_replica2},null={responseHeader={status=0,QTime=6319},core=nodes_used_collection_shard1_replica2},null={responseHeader={status=0,QTime=6384},core=nodes_used_collection_shard1_replica1}}}
[junit4:junit4] 2> 2131285 T4584 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2} status=0 QTime=6464
[junit4:junit4] 2> 2131286 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 2131288 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2131302 T4649 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@406a3841 name:ZooKeeperConnection Watcher:127.0.0.1:50532/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2131302 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2131305 T4551 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2131314 T4551 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 2131317 T4551 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 2131329 T4651 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d8b5e6a name:ZooKeeperConnection Watcher:127.0.0.1:50532/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 2131329 T4551 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 2131332 T4551 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 2131361 T4585 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=4&maxShardsPerNode=2&name=awholynewcollection_0&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 2131365 T4573 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 2131368 T4573 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 2131370 T4575 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"4",
[junit4:junit4] 2> "maxShardsPerNode":"2",
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"awholynewcollection_0",
[junit4:junit4] 2> "replicationFactor":"2"}
[junit4:junit4] 2> 2131371 T4575 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica1 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50542_
[junit4:junit4] 2> 2131371 T4575 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica2 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50545_
[junit4:junit4] 2> 2131371 T4575 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica1 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50548_
[junit4:junit4] 2> 2131372 T4575 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica2 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50539_
[junit4:junit4] 2> 2131372 T4575 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica1 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50535_
[junit4:junit4] 2> 2131373 T4575 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica2 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50542_
[junit4:junit4] 2> 2131390 T4568 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/awholynewcollection_0_shard3_replica1
[junit4:junit4] 2> 2131389 T4627 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/awholynewcollection_0_shard2_replica1
[junit4:junit4] 2> 2131394 T4627 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2131388 T4615 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard1_replica2
[junit4:junit4] 2> 2131396 T4615 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2131386 T4587 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/awholynewcollection_0_shard2_replica2
[junit4:junit4] 2> 2131398 T4587 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2131393 T4575 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica1 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:50545_
[junit4:junit4] 2> 2131393 T4568 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2131401 T4596 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard3_replica2
[junit4:junit4] 2> 2131401 T4596 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2131393 T4601 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard1_replica1
[junit4:junit4] 2> 2131407 T4601 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2131403 T4615 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 2131407 T4615 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2131403 T4627 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 2131409 T4627 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2131401 T4575 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica2 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:50548_
[junit4:junit4] 2> 2131409 T4601 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 2131412 T4601 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2131408 T4568 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 2131408 T4611 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard4_replica1
[junit4:junit4] 2> 2131414 T4611 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2131415 T4615 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2131417 T4615 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 2131407 T4587 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 2131419 T4587 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2131407 T4596 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 2131421 T4596 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2131422 T4627 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2131422 T4627 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 2131418 T4624 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/awholynewcollection_0_shard4_replica2
[junit4:junit4] 2> 2131424 T4624 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 2131414 T4568 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2131426 T4601 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2131427 T4601 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 2131429 T4624 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 2131429 T4624 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2131421 T4611 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 2131431 T4611 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 2131431 T4596 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 2131433 T4615 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2131431 T4587 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 2131434 T4587 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 2131437 T4568 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 2131439 T4627 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2131440 T4611 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 2131442 T4624 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/awholynewcollection_0_shard4_replica2/'
[junit4:junit4] 2> 2131444 T4615 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard1_replica2/'
[junit4:junit4] 2> 2131449 T4601 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2131450 T4611 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2131470 T4596 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2131471 T4587 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2131472 T4627 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/awholynewcollection_0_shard2_replica1/'
[junit4:junit4] 2> 2131488 T4611 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard4_replica1/'
[junit4:junit4] 2> 2131492 T4601 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard1_replica1/'
[junit4:junit4] 2> 2131509 T4587 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/awholynewcollection_0_shard2_replica2/'
[junit4:junit4] 2> 2131516 T4596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard3_replica2/'
[junit4:junit4] 2> 2131521 T4568 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 2131558 T4568 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/awholynewcollection_0_shard3_replica1/'
[junit4:junit4] 2> 2131794 T4627 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2131845 T4568 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2131863 T4611 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2131948 T4624 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2131971 T4601 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2131996 T4615 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2132037 T4596 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2132125 T4587 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 2132330 T4624 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2132350 T4611 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2132354 T4624 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2132358 T4611 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2132366 T4611 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica1] Schema name=test
[junit4:junit4] 2> 2132384 T4627 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2132399 T4624 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica2] Schema name=test
[junit4:junit4] 2> 2132408 T4627 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2132435 T4568 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2132475 T4568 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2132483 T4568 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica1] Schema name=test
[junit4:junit4] 2> 2132492 T4596 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2132494 T4627 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica1] Schema name=test
[junit4:junit4] 2> 2132498 T4596 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2132543 T4601 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2132551 T4596 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica2] Schema name=test
[junit4:junit4] 2> 2132561 T4601 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2132589 T4615 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2132597 T4601 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica1] Schema name=test
[junit4:junit4] 2> 2132634 T4615 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2132652 T4615 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica2] Schema name=test
[junit4:junit4] 2> 2132638 T4587 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 2132669 T4587 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 2132677 T4574 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2132684 T4574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50539__nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50539_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50539"}
[junit4:junit4] 2> 2132704 T4574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50545__nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50545_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50545"}
[junit4:junit4] 2> 2132717 T4587 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica2] Schema name=test
[junit4:junit4] 2> 2132747 T4574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50548__nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50548_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50548"}
[junit4:junit4] 2> 2133137 T4574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50535__nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50535_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50535"}
[junit4:junit4] 2> 2133192 T4578 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2133193 T4591 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2133193 T4573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2133194 T4649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2133192 T4605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2133193 T4619 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2133193 T4633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2133193 T4651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2133281 T4626 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:50535__nodes_used_collection_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 2133281 T4626 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:50535__nodes_used_collection_shard1_replica1&state=recovering&nodeName=127.0.0.1:50535_&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard1_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2004
[junit4:junit4] 2> 2134209 T4614 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:50539__nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 3 seconds.
[junit4:junit4] 2> 2134210 T4614 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:50539__nodes_used_collection_shard2_replica1&state=recovering&nodeName=127.0.0.1:50539_&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard2_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=3015
[junit4:junit4] 2> ASYNC NEW_CORE C3397 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@2b2e9560 url=http://127.0.0.1:50535/nodes_used_collection_shard1_replica1 node=127.0.0.1:50535_ C3397_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50535_, base_url=http://127.0.0.1:50535}
[junit4:junit4] 2> 2135284 T4647 C3397 P50535 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50548/nodes_used_collection_shard1_replica2/ core=nodes_used_collection_shard1_replica1 - recoveringAfterStartup=true
[junit4:junit4] 2> 2135285 T4647 C3397 P50535 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:50535 START replicas=[http://127.0.0.1:50548/nodes_used_collection_shard1_replica2/] nUpdates=100
[junit4:junit4] 2> 2135286 T4647 C3397 P50535 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 2135286 T4647 C3397 P50535 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2135286 T4647 C3397 P50535 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2135286 T4647 C3397 P50535 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2135292 T4647 C3397 P50535 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 2135292 T4647 C3397 P50535 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50548/nodes_used_collection_shard1_replica2/. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2135293 T4647 C3397 P50535 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C3398 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@524dd756 url=http://127.0.0.1:50548/nodes_used_collection_shard1_replica2 node=127.0.0.1:50548_ C3398_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50548_, base_url=http://127.0.0.1:50548, leader=true}
[junit4:junit4] 2> 2135369 T4625 C3398 P50548 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> ASYNC NEW_CORE C3399 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@524dd756 url=http://127.0.0.1:50548/nodes_used_collection_shard1_replica2 node=127.0.0.1:50548_ C3399_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50548_, base_url=http://127.0.0.1:50548, leader=true}
[junit4:junit4] 2> 2135463 T4626 C3399 P50548 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2135478 T4626 C3399 P50548 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16581dd7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a50e5c7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2135479 T4626 C3399 P50548 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2135480 T4626 C3399 P50548 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16581dd7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a50e5c7),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@16581dd7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2a50e5c7),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2135480 T4626 C3399 P50548 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2135481 T4626 C3399 P50548 oass.SolrIndexSearcher.<init> Opening Searcher@621b040d realtime
[junit4:junit4] 2> 2135482 T4626 C3399 P50548 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2135482 T4626 C3399 P50548 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard1_replica2] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 19
[junit4:junit4] 2> ASYNC NEW_CORE C3400 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@2b2e9560 url=http://127.0.0.1:50535/nodes_used_collection_shard1_replica1 node=127.0.0.1:50535_ C3400_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50535_, base_url=http://127.0.0.1:50535}
[junit4:junit4] 2> 2135484 T4647 C3400 P50535 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2135484 T4647 C3400 P50535 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 2135504 T4629 C3399 P50548 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2135505 T4629 C3399 P50548 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=7
[junit4:junit4] 2> 2135515 T4647 C3400 P50535 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 2135515 T4647 C3400 P50535 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 2135516 T4647 C3400 P50535 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 2135529 T4625 C3399 P50548 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=9
[junit4:junit4] 2> 2135531 T4647 C3400 P50535 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 2135539 T4647 C3400 P50535 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/data/index.20130529151803062
[junit4:junit4] 2> 2135540 T4647 C3400 P50535 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@a2d2c9d lockFactory=org.apache.lucene.store.NativeFSLockFactory@462fc8de) fullCopy=false
[junit4:junit4] 2> ASYNC NEW_CORE C3401 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@524dd756 url=http://127.0.0.1:50548/nodes_used_collection_shard1_replica2 node=127.0.0.1:50548_ C3401_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50548_, base_url=http://127.0.0.1:50548, leader=true}
[junit4:junit4] 2> 2135563 T4628 C3401 P50548 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C3402 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@2b2e9560 url=http://127.0.0.1:50535/nodes_used_collection_shard1_replica1 node=127.0.0.1:50535_ C3402_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50535_, base_url=http://127.0.0.1:50535}
[junit4:junit4] 2> 2135577 T4647 C3402 P50535 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 2135584 T4647 C3402 P50535 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 2135584 T4647 C3402 P50535 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2135589 T4647 C3402 P50535 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@714cf3aa lockFactory=org.apache.lucene.store.NativeFSLockFactory@2130ede),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@714cf3aa lockFactory=org.apache.lucene.store.NativeFSLockFactory@2130ede),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2135600 T4647 C3402 P50535 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2135601 T4647 C3402 P50535 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 2135601 T4647 C3402 P50535 oass.SolrIndexSearcher.<init> Opening Searcher@545bf18d main
[junit4:junit4] 2> 2135603 T4643 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@545bf18d main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 2135603 T4647 C3402 P50535 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/data/index.20130529151803062 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/data/index.20130529151803062;done=true>>]
[junit4:junit4] 2> 2135604 T4647 C3402 P50535 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/data/index.20130529151803062
[junit4:junit4] 2> 2135604 T4647 C3402 P50535 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/nodes_used_collection_shard1_replica1/data/index.20130529151803062
[junit4:junit4] 2> 2135604 T4647 C3402 P50535 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2135605 T4647 C3402 P50535 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 2135605 T4647 C3402 P50535 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=active
[junit4:junit4] 2> 2135609 T4647 C3402 P50535 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> ASYNC NEW_CORE C3403 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@78ea44c url=http://127.0.0.1:50539/nodes_used_collection_shard2_replica1 node=127.0.0.1:50539_ C3403_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50539_, base_url=http://127.0.0.1:50539}
[junit4:junit4] 2> 2136223 T4646 C3403 P50539 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50545/nodes_used_collection_shard2_replica2/ core=nodes_used_collection_shard2_replica1 - recoveringAfterStartup=true
[junit4:junit4] 2> 2136224 T4646 C3403 P50539 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:50539 START replicas=[http://127.0.0.1:50545/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4:junit4] 2> 2136225 T4646 C3403 P50539 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 2136225 T4646 C3403 P50539 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2136225 T4646 C3403 P50539 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2136226 T4646 C3403 P50539 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2136226 T4646 C3403 P50539 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 2136226 T4646 C3403 P50539 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50545/nodes_used_collection_shard2_replica2/. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2136229 T4646 C3403 P50539 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C3404 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@4744f941 url=http://127.0.0.1:50545/nodes_used_collection_shard2_replica2 node=127.0.0.1:50545_ C3404_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50545_, base_url=http://127.0.0.1:50545, leader=true}
[junit4:junit4] 2> 2136239 T4610 C3404 P50545 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 2136282 T4614 C3404 P50545 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 2136346 T4614 C3404 P50545 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c13b0c1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2ae56f0c),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2136350 T4614 C3404 P50545 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2136352 T4574 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2136355 T4574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50535__nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50535_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50535"}
[junit4:junit4] 2> 2136367 T4614 C3404 P50545 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c13b0c1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2ae56f0c),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4c13b0c1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2ae56f0c),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2136369 T4614 C3404 P50545 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2136378 T4605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2136379 T4633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2136380 T4651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2136380 T4591 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2136380 T4649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2136381 T4619 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2136378 T4578 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2136379 T4573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2136383 T4614 C3404 P50545 oass.SolrIndexSearcher.<init> Opening Searcher@3bc06419 realtime
[junit4:junit4] 2> 2136388 T4614 C3404 P50545 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 2136391 T4614 C3404 P50545 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard2_replica2] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 109
[junit4:junit4] 2> 2136392 T4646 C3403 P50539 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2136393 T4646 C3403 P50539 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 2136399 T4613 C3404 P50545 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 2136400 T4613 C3404 P50545 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=3
[junit4:junit4] 2> 2136411 T4646 C3403 P50539 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 2136411 T4646 C3403 P50539 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 2136412 T4646 C3403 P50539 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 2136435 T4610 C3404 P50545 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=20
[junit4:junit4] 2> 2136437 T4646 C3403 P50539 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 2136460 T4646 C3403 P50539 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/data/index.20130529151803968
[junit4:junit4] 2> 2136472 T4646 C3403 P50539 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@25555fec lockFactory=org.apache.lucene.store.NativeFSLockFactory@2fba7976) fullCopy=false
[junit4:junit4] 2> 2136476 T4612 C3404 P50545 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> ASYNC NEW_CORE C3405 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@78ea44c url=http://127.0.0.1:50539/nodes_used_collection_shard2_replica1 node=127.0.0.1:50539_ C3405_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50539_, base_url=http://127.0.0.1:50539}
[junit4:junit4] 2> 2136505 T4646 C3405 P50539 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 2136510 T4646 C3405 P50539 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 2136511 T4646 C3405 P50539 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2136513 T4646 C3405 P50539 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c38e69e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5772d05a),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c38e69e lockFactory=org.apache.lucene.store.NativeFSLockFactory@5772d05a),segFN=segments_2,generation=2}
[junit4:junit4] 2> 2136514 T4646 C3405 P50539 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 2136514 T4646 C3405 P50539 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 2136515 T4646 C3405 P50539 oass.SolrIndexSearcher.<init> Opening Searcher@149fddbb main
[junit4:junit4] 2> 2136517 T4642 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@149fddbb main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 2136524 T4646 C3405 P50539 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/data/index.20130529151803968 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/data/index.20130529151803968;done=true>>]
[junit4:junit4] 2> 2136526 T4646 C3405 P50539 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/data/index.20130529151803968
[junit4:junit4] 2> 2136538 T4646 C3405 P50539 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/nodes_used_collection_shard2_replica1/data/index.20130529151803968
[junit4:junit4] 2> 2136539 T4646 C3405 P50539 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2136539 T4646 C3405 P50539 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2136539 T4646 C3405 P50539 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=active
[junit4:junit4] 2> 2136549 T4646 C3405 P50539 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 2137704 T4587 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2137707 T4627 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2137808 T4587 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2137823 T4587 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2137834 T4596 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2137858 T4627 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2137865 T4627 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2137890 T4611 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2137907 T4596 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2137929 T4574 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2137929 T4596 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2137931 T4574 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50539__nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50539_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50539"}
[junit4:junit4] 2> 2137946 T4605 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2137946 T4573 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2137947 T4649 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2137947 T4619 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2137947 T4633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2137948 T4651 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2137949 T4591 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2137947 T4578 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2137964 T4611 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2138016 T4611 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2138031 T4568 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2138081 T4568 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2138095 T4587 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138101 T4568 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2138104 T4601 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2138106 T4627 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138130 T4587 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138133 T4596 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138139 T4627 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138171 T4615 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2138182 T4587 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138183 T4601 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2138202 T4611 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138202 T4587 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138204 T4587 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138204 T4587 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138208 T4601 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2138213 T4587 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138213 T4587 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138214 T4587 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2138214 T4587 oasc.SolrCore.<init> [awholynewcollection_0_shard2_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/awholynewcollection_0_shard2_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/awholynewcollection_0_shard2_replica2/data/
[junit4:junit4] 2> 2138215 T4587 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@23441f2b
[junit4:junit4] 2> 2138216 T4587 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2138218 T4587 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/awholynewcollection_0_shard2_replica2/data
[junit4:junit4] 2> 2138219 T4587 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/awholynewcollection_0_shard2_replica2/data/index/
[junit4:junit4] 2> 2138219 T4587 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard2_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/awholynewcollection_0_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2138220 T4587 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/awholynewcollection_0_shard2_replica2/data/index
[junit4:junit4] 2> 2138223 T4611 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138228 T4627 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138224 T4596 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138246 T4627 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138246 T4627 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138247 T4627 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138252 T4611 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138260 T4596 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138261 T4627 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138262 T4627 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138262 T4627 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2138263 T4627 oasc.SolrCore.<init> [awholynewcollection_0_shard2_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/awholynewcollection_0_shard2_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/awholynewcollection_0_shard2_replica1/data/
[junit4:junit4] 2> 2138299 T4627 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@23441f2b
[junit4:junit4] 2> 2138300 T4627 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2138268 T4615 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2138303 T4611 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138304 T4611 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138305 T4611 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138308 T4615 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2138314 T4568 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138317 T4627 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/awholynewcollection_0_shard2_replica1/data
[junit4:junit4] 2> 2138318 T4627 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/awholynewcollection_0_shard2_replica1/data/index/
[junit4:junit4] 2> 2138319 T4627 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard2_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/awholynewcollection_0_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2138319 T4596 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138322 T4596 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138323 T4596 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138321 T4627 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/awholynewcollection_0_shard2_replica1/data/index
[junit4:junit4] 2> 2138325 T4587 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@31c9ff62 lockFactory=org.apache.lucene.store.NativeFSLockFactory@607c43a2),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2138328 T4611 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138329 T4611 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138329 T4611 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2138330 T4568 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138333 T4596 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138333 T4596 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138334 T4596 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2138335 T4596 oasc.SolrCore.<init> [awholynewcollection_0_shard3_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard3_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard3_replica2/data/
[junit4:junit4] 2> 2138335 T4596 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@23441f2b
[junit4:junit4] 2> 2138336 T4596 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2138326 T4624 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2138330 T4611 oasc.SolrCore.<init> [awholynewcollection_0_shard4_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard4_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard4_replica1/data/
[junit4:junit4] 2> 2138342 T4611 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@23441f2b
[junit4:junit4] 2> 2138344 T4611 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2138328 T4587 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2138349 T4596 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard3_replica2/data
[junit4:junit4] 2> 2138350 T4596 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard3_replica2/data/index/
[junit4:junit4] 2> 2138360 T4596 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard3_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard3_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2138361 T4596 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard3_replica2/data/index
[junit4:junit4] 2> 2138362 T4611 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard4_replica1/data
[junit4:junit4] 2> 2138364 T4611 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard4_replica1/data/index/
[junit4:junit4] 2> 2138364 T4611 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard4_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard4_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2138365 T4611 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard4_replica1/data/index
[junit4:junit4] 2> 2138362 T4587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2138367 T4587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2138369 T4587 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2138370 T4587 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2138371 T4587 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2138371 T4587 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2138372 T4587 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2138373 T4587 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2138374 T4587 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2138371 T4627 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7266090f lockFactory=org.apache.lucene.store.NativeFSLockFactory@76bd70de),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2138376 T4627 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2138393 T4627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2138394 T4627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2138395 T4627 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2138396 T4627 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2138403 T4627 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2138404 T4627 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2138405 T4627 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2138406 T4627 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2138409 T4627 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2138412 T4596 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7c8e7553 lockFactory=org.apache.lucene.store.NativeFSLockFactory@2807cb38),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2138413 T4596 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2138421 T4611 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4145eac5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@3225772b),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2138422 T4611 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2138426 T4587 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2138429 T4596 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2138430 T4596 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2138431 T4596 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2138437 T4596 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2138441 T4611 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2138442 T4611 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2138443 T4611 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2138445 T4611 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2138446 T4611 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2138446 T4611 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2138447 T4611 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2138448 T4611 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2138452 T4611 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2138455 T4627 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2138438 T4596 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2138459 T4596 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2138460 T4596 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2138461 T4596 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2138462 T4596 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2138463 T4587 oass.SolrIndexSearcher.<init> Opening Searcher@70ee6d3e main
[junit4:junit4] 2> 2138463 T4587 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369855063374/awholynewcollection_0_shard2_replica2/data/tlog
[junit4:junit4] 2> 2138467 T4587 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2138468 T4587 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2138443 T4568 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138477 T4596 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2138475 T4611 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2138474 T4660 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard2_replica2] Registered new searcher Searcher@70ee6d3e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2138446 T4624 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2138488 T4624 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2138496 T4568 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138499 T4568 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138499 T4568 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138501 T4587 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica2 state=down
[junit4:junit4] 2> 2138507 T4627 oass.SolrIndexSearcher.<init> Opening Searcher@6bed8f83 main
[junit4:junit4] 2> 2138507 T4627 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369855069884/awholynewcollection_0_shard2_replica1/data/tlog
[junit4:junit4] 2> 2138510 T4596 oass.SolrIndexSearcher.<init> Opening Searcher@2ed2fcb4 main
[junit4:junit4] 2> 2138511 T4596 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard3_replica2/data/tlog
[junit4:junit4] 2> 2138512 T4627 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2138512 T4568 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138515 T4568 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138515 T4568 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2138516 T4568 oasc.SolrCore.<init> [awholynewcollection_0_shard3_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/awholynewcollection_0_shard3_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/awholynewcollection_0_shard3_replica1/data/
[junit4:junit4] 2> 2138516 T4568 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@23441f2b
[junit4:junit4] 2> 2138517 T4568 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2138512 T4611 oass.SolrIndexSearcher.<init> Opening Searcher@1b33bad6 main
[junit4:junit4] 2> 2138519 T4611 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard4_replica1/data/tlog
[junit4:junit4] 2> 2138520 T4568 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/awholynewcollection_0_shard3_replica1/data
[junit4:junit4] 2> 2138522 T4568 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/awholynewcollection_0_shard3_replica1/data/index/
[junit4:junit4] 2> 2138523 T4568 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard3_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/awholynewcollection_0_shard3_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2138525 T4568 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/awholynewcollection_0_shard3_replica1/data/index
[junit4:junit4] 2> 2138515 T4627 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2138515 T4596 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2138531 T4596 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2138535 T4661 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard2_replica1] Registered new searcher Searcher@6bed8f83 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2138523 T4611 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2138537 T4611 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2138540 T4601 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138541 T4663 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard3_replica2] Registered new searcher Searcher@2ed2fcb4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2138546 T4615 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138547 T4596 oasc.ZkController.publish publishing core=awholynewcollection_0_shard3_replica2 state=down
[junit4:junit4] 2> 2138579 T4611 oasc.ZkController.publish publishing core=awholynewcollection_0_shard4_replica1 state=down
[junit4:junit4] 2> 2138579 T4662 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard4_replica1] Registered new searcher Searcher@1b33bad6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2138581 T4627 oasc.ZkController.publish publishing core=awholynewcollection_0_shard2_replica1 state=down
[junit4:junit4] 2> 2138586 T4568 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@66ca5487 lockFactory=org.apache.lucene.store.NativeFSLockFactory@6c7899e7),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2138587 T4568 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2138589 T4601 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138593 T4615 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2138600 T4568 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2138600 T4568 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2138632 T4601 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138632 T4568 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2138634 T4568 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2138635 T4568 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2138635 T4568 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2138636 T4568 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2138637 T4568 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2138637 T4568 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2138635 T4615 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138659 T4601 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138659 T4601 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138660 T4601 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138665 T4568 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2138674 T4615 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138675 T4615 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138676 T4615 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2138679 T4601 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138679 T4601 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138680 T4601 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2138681 T4568 oass.SolrIndexSearcher.<init> Opening Searcher@44aecfc6 main
[junit4:junit4] 2> 2138683 T4615 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2138684 T4615 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2138684 T4568 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369855062087/awholynewcollection_0_shard3_replica1/data/tlog
[junit4:junit4] 2> 2138683 T4601 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard1_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369855065535/awholynewcollection_0_shard1_replica1/data/
[junit4:junit4] 2> 2138686 T4601 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@23441f2b
[junit4:junit4] 2> 2138687 T4568 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2138687 T4568 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2138685 T4615 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2138688 T4615 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard1_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369855067734/awholynewcollection_0_shard1_replica2/data/
[junit4:junit4] 2> 2138689 T4615 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@23441f2b
[junit4:junit4] 2> 2138690 T4615 oasc.
[...truncated too long message...]
tBase.java:126)
[junit4:junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
[junit4:junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 2402057 T4551 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 287794 T4550 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 2> 2403124 T4633 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewcollection_0/leader_elect/shard2/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 2403125 T4633 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewcollection_0/leader_elect/shard4/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> 2403126 T4633 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2403126 T4633 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 2403126 T4633 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {timestamp=Pulsing41(freqCutoff=18 minBlockSize=82 maxBlockSize=181), range_facet_l=Pulsing41(freqCutoff=18 minBlockSize=82 maxBlockSize=181), id=MockFixedIntBlock(blockSize=750), range_facet_sl=MockFixedIntBlock(blockSize=750), a_t=Pulsing41(freqCutoff=18 minBlockSize=82 maxBlockSize=181), text=PostingsFormat(name=Memory doPackFST= false), range_facet_si=Pulsing41(freqCutoff=18 minBlockSize=82 maxBlockSize=181), _version_=Pulsing41(freqCutoff=3 minBlockSize=82 maxBlockSize=181), other_tl1=Pulsing41(freqCutoff=3 minBlockSize=82 maxBlockSize=181), multiDefault=Pulsing41(freqCutoff=18 minBlockSize=82 maxBlockSize=181), a_si=Pulsing41(freqCutoff=3 minBlockSize=82 maxBlockSize=181), intDefault=Pulsing41(freqCutoff=3 minBlockSize=82 maxBlockSize=181)}, docValues:{timestamp=DocValuesFormat(name=Lucene42)}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=en_ZA, timezone=IET
[junit4:junit4] 2> NOTE: Mac OS X 10.8.3 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=1,free=59917720,total=201396224
[junit4:junit4] 2> NOTE: All tests run in this JVM: [PingRequestHandlerTest, TestHashPartitioner, TestLuceneMatchVersion, QueryEqualityTest, TestIndexSearcher, AnalysisAfterCoreReloadTest, TestFieldTypeResource, CoreContainerCoreInitFailuresTest, TestFoldingMultitermQuery, TestManagedSchema, TestPartialUpdateDeduplication, NumericFieldsTest, TermVectorComponentDistributedTest, TestMultiCoreConfBootstrap, SuggesterTSTTest, SolrRequestParserTest, ZkCLITest, ClusterStateUpdateTest, MoreLikeThisHandlerTest, StatsComponentTest, TestBM25SimilarityFactory, FastVectorHighlighterTest, TestDefaultSimilarityFactory, TestRecovery, SignatureUpdateProcessorFactoryTest, TestRTGBase, DisMaxRequestHandlerTest, TestSolrJ, AutoCommitTest, RegexBoostProcessorTest, DebugComponentTest, UUIDFieldTest, SolrPluginUtilsTest, PrimUtilsTest, PathHierarchyTokenizerFactoryTest, TestClassNameShortening, TestJmxMonitoredMap, WordBreakSolrSpellCheckerTest, TestLRUCache, TestDynamicFieldCollectionResource, OpenCloseCoreStressTest, ShowFileRequestHandlerTest, NotRequiredUniqueKeyTest, TestQueryUtils, TestOmitPositions, TestSweetSpotSimilarityFactory, DocumentAnalysisRequestHandlerTest, JsonLoaderTest, RequestHandlersTest, SimpleFacetsTest, TestSolrDeletionPolicy2, TestPerFieldSimilarity, LukeRequestHandlerTest, TestSolrIndexConfig, NoCacheHeaderTest, TestFieldCollectionResource, CopyFieldTest, ScriptEngineTest, DistanceFunctionTest, DistributedQueryElevationComponentTest, TestValueSourceCache, CurrencyFieldOpenExchangeTest, TestExtendedDismaxParser, TestIndexingPerformance, TestXIncludeConfig, SpellCheckCollatorTest, SolrIndexConfigTest, TestFastLRUCache, TestMergePolicyConfig, SortByFunctionTest, TestSchemaResource, SimplePostToolTest, SystemInfoHandlerTest, TestManagedSchemaFieldResource, TestCloudManagedSchema, ShardRoutingCustomTest, LoggingHandlerTest, FileUtilsTest, TestDefaultSearchFieldResource, SliceStateUpdateTest, SpellPossibilityIteratorTest, QueryResultKeyTest, SpellCheckComponentTest, HardAutoCommitTest, OutputWriterTest, TestSchemaNameResource, BasicZkTest, SliceStateTest, TestFuzzyAnalyzedSuggestions, CoreAdminHandlerTest, TestDocSet, ZkSolrClientTest, DistributedTermsComponentTest, TestSerializedLuceneMatchVersion, TestReplicationHandler, ShardSplitTest, TestMaxScoreQueryParser, XsltUpdateRequestHandlerTest, TestCSVResponseWriter, DistributedSpellCheckComponentTest, SpellingQueryConverterTest, DirectUpdateHandlerOptimizeTest, ReturnFieldsTest, TestShardHandlerFactory, SolrCoreCheckLockOnStartupTest, UnloadDistributedZkTest, PrimitiveFieldTypeTest, TestCollationField, RecoveryZkTest, BadCopyFieldTest, TestQuerySenderNoQuery, ChaosMonkeyShardSplitTest, TestLMDirichletSimilarityFactory, BadComponentTest, OverseerTest, TestJmxIntegration, PreAnalyzedUpdateProcessorTest, TestRandomDVFaceting, FieldMutatingUpdateProcessorTest, TestQueryTypes, DocValuesMultiTest, TestRealTimeGet, UniqFieldsUpdateProcessorFactoryTest, CachingDirectoryFactoryTest, TestElisionMultitermQuery, CacheHeaderTest, TestUpdate, TestSolrXMLSerializer, TermsComponentTest, PolyFieldTest, StatelessScriptUpdateProcessorFactoryTest, TestLFUCache, MinimalSchemaTest, SoftAutoCommitTest, TestSurroundQueryParser, TestFaceting, ExternalFileFieldSortTest, TestSolrXml, SolrIndexSplitterTest, TestSolrDeletionPolicy1, LegacyHTMLStripCharFilterTest, XmlUpdateRequestHandlerTest, BasicDistributedZkTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
[junit4:junit4] Completed in 289.10s, 1 test, 1 failure <<< FAILURES!
[...truncated 447 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:380: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:360: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1240: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:884: There were test failures: 295 suites, 1232 tests, 1 failure, 18 ignored (6 assumptions)
Total time: 110 minutes 28 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 521 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/521/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
There are still nodes recoverying - waited for 230 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
at __randomizedtesting.SeedInfo.seed([2BE5CE147F33D6FF:AA03400C086CB6C3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9549 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4:junit4] 2> 1969824 T4518 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 1969834 T4518 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-CollectionsAPIDistributedZkTest-1369904871880
[junit4:junit4] 2> 1969838 T4518 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 1969841 T4519 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 1969942 T4518 oasc.ZkTestServer.run start zk server on port:50536
[junit4:junit4] 2> 1969954 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1969988 T4525 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2434ff3e name:ZooKeeperConnection Watcher:127.0.0.1:50536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1969989 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1969990 T4518 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 1970038 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1970048 T4527 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@748bd19f name:ZooKeeperConnection Watcher:127.0.0.1:50536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1970055 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1970056 T4518 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 1970110 T4518 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 1970125 T4518 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 1970136 T4518 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 1970168 T4518 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1970169 T4518 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 1970195 T4518 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 1970196 T4518 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 1970205 T4518 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1970206 T4518 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 1970219 T4518 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 1970220 T4518 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 1970252 T4518 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 1970253 T4518 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 1970261 T4518 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1970262 T4518 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 1970269 T4518 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1970271 T4518 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 1970290 T4518 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1970291 T4518 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 1970302 T4518 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1970303 T4518 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 1971023 T4518 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1971077 T4518 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50539
[junit4:junit4] 2> 1971078 T4518 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1971078 T4518 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1971079 T4518 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363
[junit4:junit4] 2> 1971079 T4518 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/solr.xml
[junit4:junit4] 2> 1971080 T4518 oasc.CoreContainer.<init> New CoreContainer 1062852302
[junit4:junit4] 2> 1971080 T4518 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/'
[junit4:junit4] 2> 1971081 T4518 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/'
[junit4:junit4] 2> 1971265 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363
[junit4:junit4] 2> 1971266 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/collection1
[junit4:junit4] 2> 1971268 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/collection1/conf
[junit4:junit4] 2> 1971463 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/collection1/conf/xslt
[junit4:junit4] 2> 1971468 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/collection1/lib
[junit4:junit4] 2> 1971470 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/collection1/lib/classes
[junit4:junit4] 2> 1971473 T4518 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/conf/core.properties, adding to cores
[junit4:junit4] 2> 1971489 T4518 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 1971490 T4518 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1971490 T4518 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 1971491 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1971491 T4518 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1971492 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1971493 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1971493 T4518 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1971494 T4518 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1971494 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 1971529 T4518 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1971530 T4518 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50536/solr
[junit4:junit4] 2> 1971530 T4518 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1971533 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1971537 T4538 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a03126b name:ZooKeeperConnection Watcher:127.0.0.1:50536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1971538 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1971544 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1971568 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1971574 T4540 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6b33e244 name:ZooKeeperConnection Watcher:127.0.0.1:50536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1971576 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1971587 T4518 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 1971596 T4518 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 1971606 T4518 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 1971612 T4518 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50539_
[junit4:junit4] 2> 1971615 T4518 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50539_
[junit4:junit4] 2> 1971625 T4518 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 1971643 T4518 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 1971654 T4518 oasc.Overseer.start Overseer (id=89778085685100547-127.0.0.1:50539_-n_0000000000) starting
[junit4:junit4] 2> 1971663 T4518 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 1971674 T4542 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 1971677 T4518 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 1971685 T4518 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 1971691 T4518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1971704 T4518 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1971705 T4518 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1971705 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1971717 T4541 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 1971735 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1971739 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1971745 T4545 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@244e8e82 name:ZooKeeperConnection Watcher:127.0.0.1:50536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1971746 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1971750 T4518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1971758 T4518 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 1972362 T4518 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1972371 T4518 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50543
[junit4:junit4] 2> 1972372 T4518 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1972372 T4518 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1972373 T4518 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803
[junit4:junit4] 2> 1972374 T4518 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/solr.xml
[junit4:junit4] 2> 1972374 T4518 oasc.CoreContainer.<init> New CoreContainer 1222729008
[junit4:junit4] 2> 1972375 T4518 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/'
[junit4:junit4] 2> 1972376 T4518 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/'
[junit4:junit4] 2> 1972553 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803
[junit4:junit4] 2> 1972555 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/collection1
[junit4:junit4] 2> 1972557 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/collection1/conf
[junit4:junit4] 2> 1972744 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/collection1/conf/xslt
[junit4:junit4] 2> 1972749 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/collection1/lib
[junit4:junit4] 2> 1972750 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/collection1/lib/classes
[junit4:junit4] 2> 1972754 T4518 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/conf/core.properties, adding to cores
[junit4:junit4] 2> 1972770 T4518 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 1972770 T4518 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1972771 T4518 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 1972771 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1972772 T4518 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1972773 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1972773 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1972774 T4518 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1972774 T4518 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1972775 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 1972810 T4518 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1972811 T4518 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50536/solr
[junit4:junit4] 2> 1972811 T4518 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1972831 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1972833 T4556 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5126a6e2 name:ZooKeeperConnection Watcher:127.0.0.1:50536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1972834 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1972840 T4520 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x13ef4b179e00005, likely client has closed socket
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
[junit4:junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
[junit4:junit4] 2> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2>
[junit4:junit4] 2> 1972840 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1972867 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1972874 T4558 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10682ba3 name:ZooKeeperConnection Watcher:127.0.0.1:50536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1972875 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1972891 T4518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1973907 T4518 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50543_
[junit4:junit4] 2> 1973910 T4518 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50543_
[junit4:junit4] 2> 1973916 T4540 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1973919 T4558 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1973922 T4545 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 1973932 T4518 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1973933 T4518 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1973934 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1974584 T4518 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1974592 T4518 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50546
[junit4:junit4] 2> 1974592 T4518 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1974593 T4518 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1974593 T4518 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005
[junit4:junit4] 2> 1974595 T4518 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/solr.xml
[junit4:junit4] 2> 1974596 T4518 oasc.CoreContainer.<init> New CoreContainer 1180518190
[junit4:junit4] 2> 1974597 T4518 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/'
[junit4:junit4] 2> 1974597 T4518 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/'
[junit4:junit4] 2> 1974773 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005
[junit4:junit4] 2> 1974775 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/collection1
[junit4:junit4] 2> 1974777 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/collection1/conf
[junit4:junit4] 2> 1974963 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/collection1/conf/xslt
[junit4:junit4] 2> 1974968 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/collection1/lib
[junit4:junit4] 2> 1974970 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/collection1/lib/classes
[junit4:junit4] 2> 1974972 T4518 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/conf/core.properties, adding to cores
[junit4:junit4] 2> 1974989 T4518 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 1974990 T4518 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1974991 T4518 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 1974991 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1974992 T4518 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1974992 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1974993 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1974993 T4518 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1974994 T4518 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1974995 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 1975026 T4518 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1975027 T4518 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50536/solr
[junit4:junit4] 2> 1975028 T4518 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1975030 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1975035 T4570 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@554093fc name:ZooKeeperConnection Watcher:127.0.0.1:50536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1975036 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1975043 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1975075 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1975086 T4572 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a8f4302 name:ZooKeeperConnection Watcher:127.0.0.1:50536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1975086 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1975097 T4518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1976111 T4518 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50546_
[junit4:junit4] 2> 1976121 T4518 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50546_
[junit4:junit4] 2> 1976130 T4572 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1976130 T4545 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1976132 T4540 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1976134 T4558 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 1976145 T4518 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1976146 T4518 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1976146 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1976792 T4518 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1976803 T4518 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50549
[junit4:junit4] 2> 1976803 T4518 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1976804 T4518 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1976805 T4518 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216
[junit4:junit4] 2> 1976805 T4518 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/solr.xml
[junit4:junit4] 2> 1976807 T4518 oasc.CoreContainer.<init> New CoreContainer 539235630
[junit4:junit4] 2> 1976808 T4518 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/'
[junit4:junit4] 2> 1976808 T4518 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/'
[junit4:junit4] 2> 1977003 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216
[junit4:junit4] 2> 1977006 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/collection1
[junit4:junit4] 2> 1977010 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/collection1/conf
[junit4:junit4] 2> 1977189 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/collection1/conf/xslt
[junit4:junit4] 2> 1977195 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/collection1/lib
[junit4:junit4] 2> 1977197 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/collection1/lib/classes
[junit4:junit4] 2> 1977199 T4518 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/conf/core.properties, adding to cores
[junit4:junit4] 2> 1977221 T4518 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 1977222 T4518 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1977222 T4518 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 1977223 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1977223 T4518 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1977224 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1977225 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1977226 T4518 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1977226 T4518 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1977227 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 1977255 T4518 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1977256 T4518 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50536/solr
[junit4:junit4] 2> 1977256 T4518 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1977259 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1977266 T4584 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@24ee9898 name:ZooKeeperConnection Watcher:127.0.0.1:50536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1977267 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1977278 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1977304 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1977309 T4586 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b7eb5b3 name:ZooKeeperConnection Watcher:127.0.0.1:50536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1977309 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1977323 T4518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1978335 T4518 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50549_
[junit4:junit4] 2> 1978344 T4518 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50549_
[junit4:junit4] 2> 1978353 T4545 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1978353 T4558 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1978354 T4586 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1978355 T4572 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1978355 T4540 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 1978368 T4518 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1978369 T4518 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1978369 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1979265 T4518 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 1979294 T4518 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:50552
[junit4:junit4] 2> 1979296 T4518 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 1979297 T4518 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 1979298 T4518 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436
[junit4:junit4] 2> 1979299 T4518 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/solr.xml
[junit4:junit4] 2> 1979299 T4518 oasc.CoreContainer.<init> New CoreContainer 1843630370
[junit4:junit4] 2> 1979300 T4518 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/'
[junit4:junit4] 2> 1979302 T4518 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/'
[junit4:junit4] 2> 1979585 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436
[junit4:junit4] 2> 1979587 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/collection1
[junit4:junit4] 2> 1979589 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/collection1/conf
[junit4:junit4] 2> 1979831 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/collection1/conf/xslt
[junit4:junit4] 2> 1979836 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/collection1/lib
[junit4:junit4] 2> 1979838 T4518 oasc.SolrCoreDiscoverer.walkFromHere Looking for cores in /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/collection1/lib/classes
[junit4:junit4] 2> 1979851 T4518 oasc.SolrCoreDiscoverer.addCore Discovered properties file /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/conf/core.properties, adding to cores
[junit4:junit4] 2> 1979879 T4518 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
[junit4:junit4] 2> 1979880 T4518 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 1979881 T4518 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
[junit4:junit4] 2> 1979881 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 1979882 T4518 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 1979883 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 1979884 T4518 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 1979884 T4518 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 1979885 T4518 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 1979885 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=0&connTimeout=0&retry=false
[junit4:junit4] 2> 1979924 T4518 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 1979925 T4518 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50536/solr
[junit4:junit4] 2> 1979925 T4518 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 1979927 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1979932 T4598 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4adee9d name:ZooKeeperConnection Watcher:127.0.0.1:50536 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1979933 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1979937 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 1979965 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1979972 T4600 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1d3c4e09 name:ZooKeeperConnection Watcher:127.0.0.1:50536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1979973 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1979985 T4518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1980994 T4518 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50552_
[junit4:junit4] 2> 1980999 T4518 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50552_
[junit4:junit4] 2> 1981008 T4540 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1981008 T4600 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1981016 T4545 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1981016 T4572 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1981020 T4558 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1981027 T4586 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 1981034 T4518 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 1981035 T4518 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 1981036 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1981058 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1981087 T4552 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 1981089 T4540 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 1981095 T4542 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "maxShardsPerNode":null,
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"nodes_used_collection",
[junit4:junit4] 2> "replicationFactor":"2"}
[junit4:junit4] 2> 1981096 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:50549_
[junit4:junit4] 2> 1981097 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:50543_
[junit4:junit4] 2> 1981098 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:50552_
[junit4:junit4] 2> 1981098 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:50546_
[junit4:junit4] 2> 1981108 T4567 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/nodes_used_collection_shard2_replica2
[junit4:junit4] 2> 1981108 T4567 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 1981114 T4553 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/nodes_used_collection_shard1_replica2
[junit4:junit4] 2> 1981115 T4553 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 1981117 T4579 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 1981117 T4579 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 1981121 T4579 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 1981121 T4579 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 1981124 T4579 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 1981124 T4579 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 1981128 T4595 oasc.CoreContainer.create Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 1981129 T4595 oasc.ZkController.createCollectionZkNode Check for collection zkNode:nodes_used_collection
[junit4:junit4] 2> 1981129 T4567 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 1981130 T4567 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 1981132 T4595 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1981133 T4595 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 1981135 T4553 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 2> 1981136 T4553 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 1981138 T4595 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/'
[junit4:junit4] 2> 1981149 T4553 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 1981150 T4579 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 1981150 T4567 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection
[junit4:junit4] 2> 1981156 T4579 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/'
[junit4:junit4] 2> 1981159 T4553 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 1981160 T4567 oasc.ZkController.readConfigName Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 2> 1981166 T4567 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/nodes_used_collection_shard2_replica2/'
[junit4:junit4] 2> 1981168 T4553 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/nodes_used_collection_shard1_replica2/'
[junit4:junit4] 2> 1981421 T4595 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1981476 T4579 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1981498 T4553 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1981557 T4567 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1981722 T4579 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1981760 T4579 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1981762 T4553 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1981769 T4553 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1981776 T4595 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1981783 T4579 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica1] Schema name=test
[junit4:junit4] 2> 1981811 T4595 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1981862 T4595 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica1] Schema name=test
[junit4:junit4] 2> 1981895 T4553 oass.IndexSchema.readSchema [nodes_used_collection_shard1_replica2] Schema name=test
[junit4:junit4] 2> 1981965 T4567 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1981983 T4567 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1982027 T4567 oass.IndexSchema.readSchema [nodes_used_collection_shard2_replica2] Schema name=test
[junit4:junit4] 2> 1985124 T4553 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1985150 T4595 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1985233 T4553 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1985238 T4595 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1985263 T4595 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1985294 T4579 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1985303 T4553 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1985356 T4579 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1985363 T4579 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1985373 T4595 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1985374 T4553 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1985384 T4595 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1985388 T4553 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1985401 T4595 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1985410 T4553 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1985412 T4595 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1985421 T4595 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1985421 T4595 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1985426 T4553 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1985433 T4553 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1985434 T4553 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1985437 T4595 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1985438 T4595 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1985439 T4595 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1985439 T4595 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/data/
[junit4:junit4] 2> 1985440 T4595 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@518e7b75
[junit4:junit4] 2> 1985441 T4595 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1985446 T4553 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1985446 T4553 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1985447 T4553 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1985448 T4553 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/nodes_used_collection_shard1_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/nodes_used_collection_shard1_replica2/data/
[junit4:junit4] 2> 1985448 T4553 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@518e7b75
[junit4:junit4] 2> 1985449 T4553 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1985454 T4595 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/data
[junit4:junit4] 2> 1985455 T4553 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/nodes_used_collection_shard1_replica2/data
[junit4:junit4] 2> 1985461 T4567 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 1985467 T4553 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/nodes_used_collection_shard1_replica2/data/index/
[junit4:junit4] 2> 1985467 T4553 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/nodes_used_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1985467 T4595 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/data/index/
[junit4:junit4] 2> 1985476 T4595 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1985478 T4553 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/nodes_used_collection_shard1_replica2/data/index
[junit4:junit4] 2> 1985479 T4567 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 1985484 T4595 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/data/index
[junit4:junit4] 2> 1985489 T4567 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 1985504 T4553 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7367648d lockFactory=org.apache.lucene.store.NativeFSLockFactory@14f96c40),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1985505 T4553 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1985510 T4553 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1985511 T4553 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1985512 T4553 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1985513 T4553 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1985513 T4553 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1985514 T4553 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1985515 T4553 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1985516 T4553 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1985516 T4553 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1985517 T4595 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@62311c97 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4197b071),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1985520 T4595 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1985522 T4579 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1985536 T4595 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1985537 T4595 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1985537 T4595 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1985538 T4595 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1985539 T4595 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1985540 T4595 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1985540 T4595 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1985541 T4595 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1985542 T4595 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1985561 T4595 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1985563 T4579 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1985556 T4553 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1985571 T4595 oass.SolrIndexSearcher.<init> Opening Searcher@1e194cc6 main
[junit4:junit4] 2> 1985571 T4595 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/data/tlog
[junit4:junit4] 2> 1985573 T4595 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1985574 T4595 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1985593 T4553 oass.SolrIndexSearcher.<init> Opening Searcher@6f2a252e main
[junit4:junit4] 2> 1985598 T4553 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/nodes_used_collection_shard1_replica2/data/tlog
[junit4:junit4] 2> 1985597 T4595 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=down
[junit4:junit4] 2> 1985594 T4606 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@1e194cc6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1985606 T4553 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1985617 T4553 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1985621 T4579 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1985628 T4607 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@6f2a252e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1985630 T4553 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=down
[junit4:junit4] 2> 1985635 T4579 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1985635 T4579 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1985636 T4579 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1985650 T4579 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1985651 T4579 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1985651 T4579 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1985652 T4579 oasc.SolrCore.<init> [nodes_used_collection_shard1_replica1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/data/
[junit4:junit4] 2> 1985653 T4579 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@518e7b75
[junit4:junit4] 2> 1985653 T4579 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1985655 T4579 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/data
[junit4:junit4] 2> 1985656 T4579 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/data/index/
[junit4:junit4] 2> 1985656 T4579 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard1_replica1] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1985657 T4579 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/data/index
[junit4:junit4] 2> 1985669 T4579 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c43084b lockFactory=org.apache.lucene.store.NativeFSLockFactory@215c3fcb),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1985670 T4579 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1985680 T4579 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1985680 T4579 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1985681 T4579 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1985682 T4579 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1985683 T4579 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1985683 T4579 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1985684 T4579 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1985685 T4579 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1985686 T4579 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1985691 T4567 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1985725 T4579 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1985758 T4579 oass.SolrIndexSearcher.<init> Opening Searcher@6f68860d main
[junit4:junit4] 2> 1985759 T4579 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/data/tlog
[junit4:junit4] 2> 1985761 T4567 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 1985761 T4579 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1985763 T4579 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1985770 T4608 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@6f68860d main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1985773 T4579 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=down
[junit4:junit4] 2> 1985783 T4567 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1985792 T4567 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1985793 T4567 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1985794 T4567 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 1985802 T4567 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 1985803 T4567 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 1985804 T4567 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1985804 T4567 oasc.SolrCore.<init> [nodes_used_collection_shard2_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/nodes_used_collection_shard2_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/nodes_used_collection_shard2_replica2/data/
[junit4:junit4] 2> 1985805 T4567 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@518e7b75
[junit4:junit4] 2> 1985806 T4567 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 1985810 T4567 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/nodes_used_collection_shard2_replica2/data
[junit4:junit4] 2> 1985813 T4567 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/nodes_used_collection_shard2_replica2/data/index/
[junit4:junit4] 2> 1985814 T4567 oasc.SolrCore.initIndex WARN [nodes_used_collection_shard2_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/nodes_used_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 1985815 T4567 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/nodes_used_collection_shard2_replica2/data/index
[junit4:junit4] 2> 1985832 T4567 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a497a3d lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fca7853),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1985834 T4567 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1985853 T4567 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 1985853 T4567 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 1985855 T4567 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 1985856 T4567 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 1985857 T4567 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 1985858 T4567 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 1985858 T4567 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 1985859 T4567 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 1985860 T4567 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 1985884 T4567 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 1985920 T4567 oass.SolrIndexSearcher.<init> Opening Searcher@a7d8810 main
[junit4:junit4] 2> 1985930 T4567 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/nodes_used_collection_shard2_replica2/data/tlog
[junit4:junit4] 2> 1985933 T4567 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 1985934 T4567 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 1985940 T4609 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@a7d8810 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 1985944 T4567 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=down
[junit4:junit4] 2> 1986812 T4541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1986813 T4541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50552_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50552"}
[junit4:junit4] 2> 1986813 T4541 oasc.Overseer$ClusterStateUpdater.createCollection Create collection nodes_used_collection with numShards 2
[junit4:junit4] 2> 1986829 T4541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50543_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50543"}
[junit4:junit4] 2> 1986853 T4541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50549_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50549"}
[junit4:junit4] 2> 1986866 T4541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50546_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50546"}
[junit4:junit4] 2> 1986897 T4540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1986898 T4572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1986898 T4600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1986899 T4545 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1986899 T4558 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1986899 T4586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1986954 T4567 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica2
[junit4:junit4] 2> 1986955 T4567 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:50546 collection:nodes_used_collection shard:shard2
[junit4:junit4] 2> 1986958 T4567 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4] 2> 1987011 T4567 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1987025 T4567 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179998
[junit4:junit4] 2> 1987617 T4595 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 1987617 T4595 oasc.ZkController.register Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:50552 collection:nodes_used_collection shard:shard2
[junit4:junit4] 2> 1987639 T4553 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica2
[junit4:junit4] 2> 1987639 T4553 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:50543 collection:nodes_used_collection shard:shard1
[junit4:junit4] 2> 1987643 T4553 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leader_elect/shard1/election
[junit4:junit4] 2> 1987668 T4553 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 1987680 T4553 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Waiting until we see more replicas up: total=2 found=1 timeoutin=179996
[junit4:junit4] 2> 1987786 T4579 oasc.CoreContainer.registerCore registering core: nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 1987787 T4579 oasc.ZkController.register Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:50549 collection:nodes_used_collection shard:shard1
[junit4:junit4] 2> 1988033 T4567 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1988034 T4567 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1988035 T4567 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50546/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 1988036 T4567 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:50546 START replicas=[http://127.0.0.1:50552/nodes_used_collection_shard2_replica1/] nUpdates=100
[junit4:junit4] 2> 1988037 T4567 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:50546 DONE. We have no versions. sync failed.
[junit4:junit4] 2> 1988038 T4567 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 1988039 T4567 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 1988039 T4567 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50546/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 1988041 T4567 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard2
[junit4:junit4] 2> ASYNC NEW_CORE C2004 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@af1401a url=http://127.0.0.1:50552/nodes_used_collection_shard2_replica1 node=127.0.0.1:50552_ C2004_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=down, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50552_, base_url=http://127.0.0.1:50552}
[junit4:junit4] 2> 1988060 T4594 C2004 P50552 oasc.SolrCore.execute [nodes_used_collection_shard2_replica1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=2
[junit4:junit4] 2> 1988187 T4553 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 1988190 T4553 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 1988190 T4553 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:50543/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 1988191 T4553 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:50543 START replicas=[http://127.0.0.1:50549/nodes_used_collection_shard1_replica1/] nUpdates=100
[junit4:junit4] 2> 1988192 T4553 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:50543 DONE. We have no versions. sync failed.
[junit4:junit4] 2> 1988192 T4553 oasc.SyncStrategy.syncReplicas Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 2> 1988193 T4553 oasc.ShardLeaderElectionContext.runLeaderProcess We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 2> 1988194 T4553 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:50543/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 1988195 T4553 oascc.SolrZkClient.makePath makePath: /collections/nodes_used_collection/leaders/shard1
[junit4:junit4] 2> ASYNC NEW_CORE C2005 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@48bcbcb3 url=http://127.0.0.1:50549/nodes_used_collection_shard1_replica1 node=127.0.0.1:50549_ C2005_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=down, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50549_, base_url=http://127.0.0.1:50549}
[junit4:junit4] 2> 1988211 T4581 C2005 P50549 oasc.SolrCore.execute [nodes_used_collection_shard1_replica1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1988446 T4541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1988508 T4572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1988509 T4558 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1988509 T4586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1988509 T4545 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1988509 T4600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1988509 T4540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1988537 T4567 oasc.ZkController.register We are http://127.0.0.1:50546/nodes_used_collection_shard2_replica2/ and leader is http://127.0.0.1:50546/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 1988538 T4567 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica2 baseURL=http://127.0.0.1:50546
[junit4:junit4] 2> 1988538 T4567 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1988539 T4567 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica2 state=active
[junit4:junit4] 2> 1988543 T4553 oasc.ZkController.register We are http://127.0.0.1:50543/nodes_used_collection_shard1_replica2/ and leader is http://127.0.0.1:50543/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 1988544 T4553 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica2 baseURL=http://127.0.0.1:50543
[junit4:junit4] 2> 1988544 T4553 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 1988545 T4553 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica2 state=active
[junit4:junit4] 2> 1988543 T4567 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1988550 T4567 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=7442
[junit4:junit4] 2> 1988550 T4553 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1988553 T4553 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica2&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=7439
[junit4:junit4] 2> 1988643 T4595 oasc.ZkController.register We are http://127.0.0.1:50552/nodes_used_collection_shard2_replica1/ and leader is http://127.0.0.1:50546/nodes_used_collection_shard2_replica2/
[junit4:junit4] 2> 1988644 T4595 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard2_replica1 baseURL=http://127.0.0.1:50552
[junit4:junit4] 2> 1988644 T4595 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 1988645 T4595 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 1988650 T4595 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1988654 T4612 C2004 P50552 oasc.RecoveryStrategy.run Starting recovery process. core=nodes_used_collection_shard2_replica1 recoveringAfterStartup=true
[junit4:junit4] 2> 1988656 T4612 C2004 P50552 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1988657 T4595 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard2&name=nodes_used_collection_shard2_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=7529
[junit4:junit4] 2> 1988658 T4612 C2004 P50552 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=recovering
[junit4:junit4] 2> 1988663 T4612 C2004 P50552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1988698 T4565 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:50552__nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1988822 T4579 oasc.ZkController.register We are http://127.0.0.1:50549/nodes_used_collection_shard1_replica1/ and leader is http://127.0.0.1:50543/nodes_used_collection_shard1_replica2/
[junit4:junit4] 2> 1988823 T4579 oasc.ZkController.register No LogReplay needed for core=nodes_used_collection_shard1_replica1 baseURL=http://127.0.0.1:50549
[junit4:junit4] 2> 1988823 T4579 oasc.ZkController.checkRecovery Core needs to recover:nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 1988825 T4579 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> 1988826 T4613 C2005 P50549 oasc.RecoveryStrategy.run Starting recovery process. core=nodes_used_collection_shard1_replica1 recoveringAfterStartup=true
[junit4:junit4] 2> 1988827 T4613 C2005 P50549 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 1988828 T4613 C2005 P50549 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=recovering
[junit4:junit4] 2> 1988830 T4579 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1988836 T4613 C2005 P50549 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1988836 T4579 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={numShards=2&shard=shard1&name=nodes_used_collection_shard1_replica1&action=CREATE&collection=nodes_used_collection&wt=javabin&qt=/admin/cores&version=2} status=0 QTime=7719
[junit4:junit4] 2> 1988849 T4542 oasc.OverseerCollectionProcessor.createCollection Finished create command on all shards for collection: nodes_used_collection
[junit4:junit4] 2> 1988876 T4549 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:50549__nodes_used_collection_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 1988885 T4558 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4] 2> 1988890 T4540 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 1988891 T4542 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=7442},core=nodes_used_collection_shard2_replica2},null={responseHeader={status=0,QTime=7439},core=nodes_used_collection_shard1_replica2},null={responseHeader={status=0,QTime=7529},core=nodes_used_collection_shard2_replica1},null={responseHeader={status=0,QTime=7719},core=nodes_used_collection_shard1_replica1}}}
[junit4:junit4] 2> 1988892 T4552 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={numShards=2&name=nodes_used_collection&replicationFactor=2&action=CREATE&wt=javabin&version=2} status=0 QTime=7806
[junit4:junit4] 2> 1988894 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1988897 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1988913 T4615 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@22ac8a46 name:ZooKeeperConnection Watcher:127.0.0.1:50536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1988914 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1988917 T4518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1988927 T4518 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 1988930 T4518 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 1988939 T4617 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2556d543 name:ZooKeeperConnection Watcher:127.0.0.1:50536/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 1988940 T4518 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 1988943 T4518 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 1988986 T4533 oasha.CollectionsHandler.handleCreateAction Creating Collection : numShards=5&maxShardsPerNode=5&name=awholynewcollection_0&replicationFactor=4&action=CREATE&wt=javabin&version=2
[junit4:junit4] 2> 1988990 T4540 oasc.DistributedQueue$LatchChildWatcher.process Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 2> 1988999 T4542 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4:junit4] 2> "operation":"createcollection",
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "maxShardsPerNode":"5",
[junit4:junit4] 2> "createNodeSet":null,
[junit4:junit4] 2> "name":"awholynewcollection_0",
[junit4:junit4] 2> "replicationFactor":"4"}
[junit4:junit4] 2> 1988999 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica1 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50552_
[junit4:junit4] 2> 1989000 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica2 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50546_
[junit4:junit4] 2> 1989001 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica3 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50549_
[junit4:junit4] 2> 1989003 T4566 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard1_replica2
[junit4:junit4] 2> 1989004 T4566 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989005 T4591 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard1_replica1
[junit4:junit4] 2> 1989005 T4591 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989007 T4566 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 1989008 T4566 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 1989005 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard1_replica4 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:50539_
[junit4:junit4] 2> 1989009 T4577 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/awholynewcollection_0_shard1_replica3
[junit4:junit4] 2> 1989010 T4577 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989008 T4591 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 1989012 T4591 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 1989014 T4577 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 1989013 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica1 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50543_
[junit4:junit4] 2> 1989015 T4566 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 1989016 T4566 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 1989015 T4577 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 1989016 T4534 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard1_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/awholynewcollection_0_shard1_replica4
[junit4:junit4] 2> 1989017 T4534 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989019 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica2 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50552_
[junit4:junit4] 2> 1989022 T4551 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/awholynewcollection_0_shard2_replica1
[junit4:junit4] 2> 1989024 T4551 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989025 T4593 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard2_replica2
[junit4:junit4] 2> 1989026 T4593 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989027 T4591 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 1989028 T4591 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 1989028 T4593 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 1989029 T4593 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 1989024 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica3 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50546_
[junit4:junit4] 2> 1989022 T4577 oasc.ZkController.getConfName Only one config set found in zk - using it:conf1
[junit4:junit4] 2> 1989031 T4577 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 1989029 T4551 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 1989032 T4551 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 1989025 T4534 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 2> 1989033 T4534 oasc.ZkController.getConfName Looking for collection configName
[junit4:junit4] 2> 1989037 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard2_replica4 as part of slice shard2 of collection awholynewcollection_0 on 127.0.0.1:50549_
[junit4:junit4] 2> 1989043 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica1 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50539_
[junit4:junit4] 2> 1989052 T4577 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989052 T4591 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989051 T4534 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 1989050 T4593 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 1989049 T4551 oascc.SolrZkClient.makePath makePath: /collections/awholynewcollection_0
[junit4:junit4] 2> 1989048 T4566 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989047 T4563 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard2_replica3
[junit4:junit4] 2> 1989058 T4563 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989046 T4578 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard2_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/awholynewcollection_0_shard2_replica4
[junit4:junit4] 2> 1989059 T4578 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989060 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica2 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50543_
[junit4:junit4] 2> 1989070 T4551 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989069 T4566 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard1_replica2/'
[junit4:junit4] 2> 1989066 T4591 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard1_replica1/'
[junit4:junit4] 2> 1989065 T4563 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989075 T4563 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989063 T4535 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/awholynewcollection_0_shard3_replica1
[junit4:junit4] 2> 1989076 T4535 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989065 T4577 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/awholynewcollection_0_shard1_replica3/'
[junit4:junit4] 2> 1989064 T4578 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989079 T4578 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989084 T4593 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989084 T4554 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/awholynewcollection_0_shard3_replica2
[junit4:junit4] 2> 1989086 T4554 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989105 T4578 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/awholynewcollection_0_shard2_replica4/'
[junit4:junit4] 2> 1989105 T4554 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989106 T4554 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989109 T4593 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard2_replica2/'
[junit4:junit4] 2> 1989115 T4534 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989116 T4535 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989116 T4535 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989117 T4563 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard2_replica3/'
[junit4:junit4] 2> 1989123 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica3 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50552_
[junit4:junit4] 2> 1989125 T4551 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/awholynewcollection_0_shard2_replica1/'
[junit4:junit4] 2> 1989129 T4554 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/awholynewcollection_0_shard3_replica2/'
[junit4:junit4] 2> 1989134 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard3_replica4 as part of slice shard3 of collection awholynewcollection_0 on 127.0.0.1:50546_
[junit4:junit4] 2> 1989136 T4594 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard3_replica3
[junit4:junit4] 2> 1989138 T4594 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989145 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica1 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:50549_
[junit4:junit4] 2> 1989148 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica2 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:50539_
[junit4:junit4] 2> 1989150 T4581 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/awholynewcollection_0_shard4_replica1
[junit4:junit4] 2> 1989152 T4581 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989154 T4564 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard3_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard3_replica4
[junit4:junit4] 2> 1989154 T4564 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989156 T4534 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/awholynewcollection_0_shard1_replica4/'
[junit4:junit4] 2> 1989157 T4581 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989157 T4581 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989161 T4581 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/awholynewcollection_0_shard4_replica1/'
[junit4:junit4] 2> 1989184 T4535 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/awholynewcollection_0_shard3_replica1/'
[junit4:junit4] 2> 1989189 T4564 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989189 T4564 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989151 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica3 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:50543_
[junit4:junit4] 2> 1989156 T4594 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989198 T4594 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989156 T4536 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/awholynewcollection_0_shard4_replica2
[junit4:junit4] 2> 1989199 T4536 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989203 T4536 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989203 T4536 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989210 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard4_replica4 as part of slice shard4 of collection awholynewcollection_0 on 127.0.0.1:50552_
[junit4:junit4] 2> 1989212 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica1 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:50546_
[junit4:junit4] 2> 1989216 T4567 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard5_replica1
[junit4:junit4] 2> 1989217 T4567 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989218 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica2 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:50549_
[junit4:junit4] 2> 1989223 T4582 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica2' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/awholynewcollection_0_shard5_replica2
[junit4:junit4] 2> 1989224 T4582 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989225 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica3 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:50539_
[junit4:junit4] 2> 1989229 T4531 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/awholynewcollection_0_shard5_replica3
[junit4:junit4] 2> 1989230 T4531 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989231 T4553 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica3' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/awholynewcollection_0_shard4_replica3
[junit4:junit4] 2> 1989269 T4553 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989253 T4567 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989270 T4567 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989253 T4594 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard3_replica3/'
[junit4:junit4] 2> 1989249 T4582 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989272 T4582 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989249 T4564 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard3_replica4/'
[junit4:junit4] 2> 1989247 T4596 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard4_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard4_replica4
[junit4:junit4] 2> 1989275 T4596 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989241 T4542 oasc.OverseerCollectionProcessor.createCollection Creating shard awholynewcollection_0_shard5_replica4 as part of slice shard5 of collection awholynewcollection_0 on 127.0.0.1:50543_
[junit4:junit4] 2> 1989284 T4531 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989285 T4531 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989286 T4553 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989286 T4553 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989284 T4596 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989287 T4596 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989293 T4552 oasc.CoreContainer.create Creating SolrCore 'awholynewcollection_0_shard5_replica4' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/awholynewcollection_0_shard5_replica4
[junit4:junit4] 2> 1989294 T4552 oasc.ZkController.createCollectionZkNode Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 2> 1989296 T4552 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 1989297 T4552 oasc.ZkController.readConfigName Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 2> 1989301 T4582 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/awholynewcollection_0_shard5_replica2/'
[junit4:junit4] 2> 1989302 T4553 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/awholynewcollection_0_shard4_replica3/'
[junit4:junit4] 2> 1989305 T4536 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/awholynewcollection_0_shard4_replica2/'
[junit4:junit4] 2> 1989301 T4531 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1369904872363/awholynewcollection_0_shard5_replica3/'
[junit4:junit4] 2> 1989308 T4567 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard5_replica1/'
[junit4:junit4] 2> 1989319 T4552 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1369904873803/awholynewcollection_0_shard5_replica4/'
[junit4:junit4] 2> 1989323 T4596 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard4_replica4/'
[junit4:junit4] 2> 1989684 T4554 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1989698 T4594 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1989819 T4551 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1989855 T4582 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1989967 T4553 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1989982 T4566 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1990005 T4552 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1990032 T4567 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1990046 T4563 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1990048 T4541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1990050 T4541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50546__nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50546_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50546"}
[junit4:junit4] 2> 1990060 T4541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50543__nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50543_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50543"}
[junit4:junit4] 2> 1990088 T4541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50552__nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50552_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50552"}
[junit4:junit4] 2> 1990144 T4534 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1990223 T4541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50549__nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50549_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50549"}
[junit4:junit4] 2> 1990281 T4615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1990283 T4572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1990288 T4600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1990288 T4586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1990288 T4558 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1990290 T4617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1990290 T4545 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1990306 T4540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1990426 T4591 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1990519 T4577 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1990664 T4535 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1990718 T4565 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:50552__nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 1990719 T4565 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:50552__nodes_used_collection_shard2_replica1&state=recovering&nodeName=127.0.0.1:50552_&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard2_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2021
[junit4:junit4] 2> 1990738 T4578 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1990826 T4596 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1990879 T4549 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:50549__nodes_used_collection_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 1990880 T4549 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:50549__nodes_used_collection_shard1_replica1&state=recovering&nodeName=127.0.0.1:50549_&action=PREPRECOVERY&checkLive=true&core=nodes_used_collection_shard1_replica2&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2004
[junit4:junit4] 2> 1991004 T4593 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1991038 T4536 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1991171 T4594 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1991176 T4594 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1991184 T4594 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica3] Schema name=test
[junit4:junit4] 2> 1991224 T4551 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1991225 T4531 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1991259 T4551 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1991276 T4581 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1991301 T4551 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica1] Schema name=test
[junit4:junit4] 2> 1991330 T4566 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1991336 T4591 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1991410 T4566 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1991421 T4591 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1991462 T4564 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 1991478 T4552 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1991482 T4566 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica2] Schema name=test
[junit4:junit4] 2> 1991492 T4552 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1991516 T4591 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica1] Schema name=test
[junit4:junit4] 2> 1991541 T4563 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1991561 T4563 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1991569 T4563 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica3] Schema name=test
[junit4:junit4] 2> 1991654 T4552 oass.IndexSchema.readSchema [awholynewcollection_0_shard5_replica4] Schema name=test
[junit4:junit4] 2> 1991675 T4553 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1991698 T4553 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1991795 T4554 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1991800 T4554 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1991939 T4582 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1991952 T4582 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1991960 T4535 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1991973 T4535 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1991974 T4554 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica2] Schema name=test
[junit4:junit4] 2> 1991995 T4535 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica1] Schema name=test
[junit4:junit4] 2> 1992079 T4553 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica3] Schema name=test
[junit4:junit4] 2> 1992116 T4582 oass.IndexSchema.readSchema [awholynewcollection_0_shard5_replica2] Schema name=test
[junit4:junit4] 2> 1992236 T4578 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1992241 T4578 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1992271 T4577 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1992279 T4577 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1992366 T4567 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1992379 T4567 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1992410 T4567 oass.IndexSchema.readSchema [awholynewcollection_0_shard5_replica1] Schema name=test
[junit4:junit4] 2> 1992403 T4577 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica3] Schema name=test
[junit4:junit4] 2> 1992557 T4593 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1992559 T4534 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1992572 T4534 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1992575 T4593 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1992582 T4536 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1992582 T4578 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica4] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C2006 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@48bcbcb3 url=http://127.0.0.1:50549/nodes_used_collection_shard1_replica1 node=127.0.0.1:50549_ C2006_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50549_, base_url=http://127.0.0.1:50549}
[junit4:junit4] 2> 1993263 T4613 C2006 P50549 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50543/nodes_used_collection_shard1_replica2/ core=nodes_used_collection_shard1_replica1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1993263 T4613 C2006 P50549 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:50549 START replicas=[http://127.0.0.1:50543/nodes_used_collection_shard1_replica2/] nUpdates=100
[junit4:junit4] 2> 1993265 T4613 C2006 P50549 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1993265 T4613 C2006 P50549 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 1993265 T4613 C2006 P50549 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 1993265 T4613 C2006 P50549 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 1993266 T4613 C2006 P50549 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1993266 T4613 C2006 P50549 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50543/nodes_used_collection_shard1_replica2/. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 1993266 T4613 C2006 P50549 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C2007 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@af1401a url=http://127.0.0.1:50552/nodes_used_collection_shard2_replica1 node=127.0.0.1:50552_ C2007_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50552_, base_url=http://127.0.0.1:50552}
[junit4:junit4] 2> 1993268 T4612 C2007 P50552 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:50546/nodes_used_collection_shard2_replica2/ core=nodes_used_collection_shard2_replica1 - recoveringAfterStartup=true
[junit4:junit4] 2> 1993271 T4612 C2007 P50552 oasu.PeerSync.sync PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:50552 START replicas=[http://127.0.0.1:50546/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4:junit4] 2> 1993272 T4612 C2007 P50552 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 1993272 T4612 C2007 P50552 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 1993273 T4612 C2007 P50552 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 1993273 T4612 C2007 P50552 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 1993273 T4612 C2007 P50552 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 1993274 T4612 C2007 P50552 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:50546/nodes_used_collection_shard2_replica2/. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 1993274 T4612 C2007 P50552 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 1993287 T4536 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> ASYNC NEW_CORE C2008 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@55fc877e url=http://127.0.0.1:50543/nodes_used_collection_shard1_replica2 node=127.0.0.1:50543_ C2008_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50543_, base_url=http://127.0.0.1:50543, leader=true}
[junit4:junit4] 2> 1993294 T4550 C2008 P50543 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=0
[junit4:junit4] 2> 1993296 T4593 oass.IndexSchema.readSchema [awholynewcollection_0_shard2_replica2] Schema name=test
[junit4:junit4] 2> 1993295 T4536 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica2] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C2009 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@49ae09a9 url=http://127.0.0.1:50546/nodes_used_collection_shard2_replica2 node=127.0.0.1:50546_ C2009_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50546_, base_url=http://127.0.0.1:50546, leader=true}
[junit4:junit4] 2> 1993326 T4568 C2009 P50546 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=1
[junit4:junit4] 2> 1993332 T4531 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1993352 T4531 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1993355 T4565 C2009 P50546 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1993380 T4565 C2009 P50546 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a497a3d lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fca7853),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1993381 T4565 C2009 P50546 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1993387 T4531 oass.IndexSchema.readSchema [awholynewcollection_0_shard5_replica3] Schema name=test
[junit4:junit4] 2> 1993408 T4550 C2008 P50543 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 1993433 T4550 C2008 P50543 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7367648d lockFactory=org.apache.lucene.store.NativeFSLockFactory@14f96c40),segFN=segments_1,generation=1}
[junit4:junit4] 2> 1993437 T4550 C2008 P50543 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 1993467 T4550 C2008 P50543 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7367648d lockFactory=org.apache.lucene.store.NativeFSLockFactory@14f96c40),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@7367648d lockFactory=org.apache.lucene.store.NativeFSLockFactory@14f96c40),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1993468 T4550 C2008 P50543 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> ASYNC NEW_CORE C2010 name=nodes_used_collection_shard2_replica2 org.apache.solr.core.SolrCore@49ae09a9 url=http://127.0.0.1:50546/nodes_used_collection_shard2_replica2 node=127.0.0.1:50546_ C2010_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica2 props:{shard=shard2, state=active, core=nodes_used_collection_shard2_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50546_, base_url=http://127.0.0.1:50546, leader=true}
[junit4:junit4] 2> 1993542 T4565 C2010 P50546 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a497a3d lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fca7853),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a497a3d lockFactory=org.apache.lucene.store.NativeFSLockFactory@5fca7853),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1993543 T4565 C2010 P50546 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> ASYNC NEW_CORE C2011 name=nodes_used_collection_shard1_replica2 org.apache.solr.core.SolrCore@55fc877e url=http://127.0.0.1:50543/nodes_used_collection_shard1_replica2 node=127.0.0.1:50543_ C2011_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica2 props:{shard=shard1, state=active, core=nodes_used_collection_shard1_replica2, collection=nodes_used_collection, node_name=127.0.0.1:50543_, base_url=http://127.0.0.1:50543, leader=true}
[junit4:junit4] 2> 1993542 T4550 C2011 P50543 oass.SolrIndexSearcher.<init> Opening Searcher@b3fa785 realtime
[junit4:junit4] 2> 1993545 T4550 C2011 P50543 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1993545 T4550 C2011 P50543 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard1_replica2] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 138
[junit4:junit4] 2> 1993550 T4534 oass.IndexSchema.readSchema [awholynewcollection_0_shard1_replica4] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C2012 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@48bcbcb3 url=http://127.0.0.1:50549/nodes_used_collection_shard1_replica1 node=127.0.0.1:50549_ C2012_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50549_, base_url=http://127.0.0.1:50549}
[junit4:junit4] 2> 1993565 T4613 C2012 P50549 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1993573 T4613 C2012 P50549 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1993577 T4550 C2011 P50543 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1993578 T4550 C2011 P50543 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 1993580 T4613 C2012 P50549 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1993580 T4613 C2012 P50549 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1993581 T4613 C2012 P50549 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1993593 T4565 C2010 P50546 oass.SolrIndexSearcher.<init> Opening Searcher@21865407 realtime
[junit4:junit4] 2> 1993594 T4565 C2010 P50546 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 1993594 T4565 C2010 P50546 oasup.LogUpdateProcessor.finish [nodes_used_collection_shard2_replica2] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 239
[junit4:junit4] 2> ASYNC NEW_CORE C2013 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@af1401a url=http://127.0.0.1:50552/nodes_used_collection_shard2_replica1 node=127.0.0.1:50552_ C2013_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50552_, base_url=http://127.0.0.1:50552}
[junit4:junit4] 2> 1993598 T4612 C2013 P50552 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1993598 T4612 C2013 P50552 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 1993609 T4565 C2010 P50546 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 1993609 T4565 C2010 P50546 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 2> 1993627 T4612 C2013 P50552 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 1993628 T4612 C2013 P50552 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 1993628 T4612 C2013 P50552 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 1993654 T4549 C2011 P50543 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=45
[junit4:junit4] 2> 1993656 T4613 C2012 P50549 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1993667 T4568 C2010 P50546 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=32
[junit4:junit4] 2> 1993668 T4612 C2013 P50552 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 1993669 T4613 C2012 P50549 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/data/index.20130530040815701
[junit4:junit4] 2> 1993671 T4613 C2012 P50549 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@1ba98881 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1294e0bc) fullCopy=false
[junit4:junit4] 2> 1993674 T4612 C2013 P50552 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/data/index.20130530040815713
[junit4:junit4] 2> 1993674 T4612 C2013 P50552 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@461b6360 lockFactory=org.apache.lucene.store.NativeFSLockFactory@58abde89) fullCopy=false
[junit4:junit4] 2> 1993693 T4565 C2010 P50546 oasc.SolrCore.execute [nodes_used_collection_shard2_replica2] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1993694 T4550 C2011 P50543 oasc.SolrCore.execute [nodes_used_collection_shard1_replica2] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 1993711 T4613 C2012 P50549 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1993711 T4612 C2013 P50552 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 1993797 T4596 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1993801 T4596 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1993807 T4581 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1993824 T4581 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> ASYNC NEW_CORE C2014 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@48bcbcb3 url=http://127.0.0.1:50549/nodes_used_collection_shard1_replica1 node=127.0.0.1:50549_ C2014_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50549_, base_url=http://127.0.0.1:50549}
[junit4:junit4] 2> 1993836 T4613 C2014 P50549 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1993837 T4613 C2014 P50549 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 1993839 T4581 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C2015 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@af1401a url=http://127.0.0.1:50552/nodes_used_collection_shard2_replica1 node=127.0.0.1:50552_ C2015_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50552_, base_url=http://127.0.0.1:50552}
[junit4:junit4] 2> 1993846 T4612 C2015 P50552 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 1993847 T4612 C2015 P50552 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 1993901 T4596 oass.IndexSchema.readSchema [awholynewcollection_0_shard4_replica4] Schema name=test
[junit4:junit4] 2> 1993904 T4612 C2015 P50552 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@62311c97 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4197b071),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@62311c97 lockFactory=org.apache.lucene.store.NativeFSLockFactory@4197b071),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1993905 T4612 C2015 P50552 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1993906 T4613 C2014 P50549 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c43084b lockFactory=org.apache.lucene.store.NativeFSLockFactory@215c3fcb),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1c43084b lockFactory=org.apache.lucene.store.NativeFSLockFactory@215c3fcb),segFN=segments_2,generation=2}
[junit4:junit4] 2> 1993906 T4613 C2014 P50549 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 1993921 T4612 C2015 P50552 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1993921 T4613 C2014 P50549 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 1993937 T4612 C2015 P50552 oass.SolrIndexSearcher.<init> Opening Searcher@6b888518 main
[junit4:junit4] 2> 1993939 T4606 oasc.SolrCore.registerSearcher [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@6b888518 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1993940 T4613 C2014 P50549 oass.SolrIndexSearcher.<init> Opening Searcher@79e1c5fd main
[junit4:junit4] 2> 1993942 T4608 oasc.SolrCore.registerSearcher [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@79e1c5fd main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 1993950 T4612 C2015 P50552 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/data/index.20130530040815713 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/data/index.20130530040815713;done=true>>]
[junit4:junit4] 2> 1993951 T4612 C2015 P50552 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/data/index.20130530040815713
[junit4:junit4] 2> 1993951 T4613 C2014 P50549 oasc.CachingDirectoryFactory.closeCacheValue looking to close /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/data/index.20130530040815701 [CachedDir<<refCount=0;path=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/data/index.20130530040815701;done=true>>]
[junit4:junit4] 2> 1993952 T4613 C2014 P50549 oasc.CachingDirectoryFactory.close Closing directory: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/data/index.20130530040815701
[junit4:junit4] 2> ASYNC NEW_CORE C2016 name=nodes_used_collection_shard1_replica1 org.apache.solr.core.SolrCore@48bcbcb3 url=http://127.0.0.1:50549/nodes_used_collection_shard1_replica1 node=127.0.0.1:50549_ C2016_STATE=coll:nodes_used_collection core:nodes_used_collection_shard1_replica1 props:{shard=shard1, state=recovering, core=nodes_used_collection_shard1_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50549_, base_url=http://127.0.0.1:50549}
[junit4:junit4] 2> 1993986 T4613 C2016 P50549 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1369904878216/nodes_used_collection_shard1_replica1/data/index.20130530040815701
[junit4:junit4] 2> 1993987 T4613 C2016 P50549 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 1993987 T4613 C2016 P50549 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 1993987 T4613 C2016 P50549 oasc.ZkController.publish publishing core=nodes_used_collection_shard1_replica1 state=active
[junit4:junit4] 2> ASYNC NEW_CORE C2017 name=nodes_used_collection_shard2_replica1 org.apache.solr.core.SolrCore@af1401a url=http://127.0.0.1:50552/nodes_used_collection_shard2_replica1 node=127.0.0.1:50552_ C2017_STATE=coll:nodes_used_collection core:nodes_used_collection_shard2_replica1 props:{shard=shard2, state=recovering, core=nodes_used_collection_shard2_replica1, collection=nodes_used_collection, node_name=127.0.0.1:50552_, base_url=http://127.0.0.1:50552}
[junit4:junit4] 2> 1993987 T4612 C2017 P50552 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/nodes_used_collection_shard2_replica1/data/index.20130530040815713
[junit4:junit4] 2> 1993989 T4612 C2017 P50552 oasc.RecoveryStrategy.replay No replay needed. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 1993989 T4612 C2017 P50552 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 1993990 T4612 C2017 P50552 oasc.ZkController.publish publishing core=nodes_used_collection_shard2_replica1 state=active
[junit4:junit4] 2> 1993990 T4613 C2016 P50549 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard1_replica1
[junit4:junit4] 2> 1994002 T4612 C2017 P50552 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 2> 1994009 T4564 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 1994034 T4564 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 1994253 T4564 oass.IndexSchema.readSchema [awholynewcollection_0_shard3_replica4] Schema name=test
[junit4:junit4] 2> 1994944 T4541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 1994946 T4541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50549__nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50549_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50549"}
[junit4:junit4] 2> 1994958 T4541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:50552__nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 2> "collection":"nodes_used_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50552_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50552"}
[junit4:junit4] 2> 1994989 T4572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1994989 T4600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1994990 T4545 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1994989 T4615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1994990 T4617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1994990 T4586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1994990 T4558 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 1994999 T4540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2005140 T4566 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2005427 T4594 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2005493 T4566 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2005629 T4594 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2005636 T4594 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2005685 T4566 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2005905 T4566 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2005940 T4566 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2005953 T4594 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2005967 T4594 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2006073 T4577 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2006104 T4566 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2006110 T4566 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2006110 T4566 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2006111 T4566 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2006128 T4566 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2006129 T4566 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2006129 T4566 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2006130 T4566 oasc.SolrCore.<init> [awholynewcollection_0_shard1_replica2] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard1_replica2/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard1_replica2/data/
[junit4:junit4] 2> 2006130 T4566 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@518e7b75
[junit4:junit4] 2> 2006131 T4566 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2006147 T4566 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard1_replica2/data
[junit4:junit4] 2> 2006174 T4566 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard1_replica2/data/index/
[junit4:junit4] 2> 2006174 T4566 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard1_replica2] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2006186 T4566 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard1_replica2/data/index
[junit4:junit4] 2> 2006227 T4577 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2006312 T4566 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@10f28098 lockFactory=org.apache.lucene.store.NativeFSLockFactory@456b52e0),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2006312 T4566 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2006334 T4594 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2006349 T4594 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2006349 T4594 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2006350 T4594 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 2> 2006354 T4594 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
[junit4:junit4] 2> 2006354 T4594 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
[junit4:junit4] 2> 2006355 T4594 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2006356 T4594 oasc.SolrCore.<init> [awholynewcollection_0_shard3_replica3] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard3_replica3/, dataDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard3_replica3/data/
[junit4:junit4] 2> 2006356 T4594 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@518e7b75
[junit4:junit4] 2> 2006357 T4577 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2006357 T4594 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 2006361 T4594 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard3_replica3/data
[junit4:junit4] 2> 2006367 T4594 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard3_replica3/data/index/
[junit4:junit4] 2> 2006367 T4594 oasc.SolrCore.initIndex WARN [awholynewcollection_0_shard3_replica3] Solr index directory '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard3_replica3/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 2006383 T4594 oasc.CachingDirectoryFactory.get return new directory for /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard3_replica3/data/index
[junit4:junit4] 2> 2006432 T4594 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@acd8bee lockFactory=org.apache.lucene.store.NativeFSLockFactory@118b2e25),segFN=segments_1,generation=1}
[junit4:junit4] 2> 2006432 T4594 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 2006448 T4594 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2006449 T4594 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2006450 T4594 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2006451 T4594 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2006452 T4594 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2006452 T4594 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2006453 T4594 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2006454 T4594 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2006485 T4594 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2006504 T4566 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 2006505 T4566 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 2006506 T4566 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 2006507 T4566 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 2006508 T4566 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 2006508 T4566 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 2006509 T4566 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 2006510 T4566 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 2006511 T4566 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 2006615 T4577 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2006625 T4566 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2006733 T4581 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2006782 T4566 oass.SolrIndexSearcher.<init> Opening Searcher@53fd221e main
[junit4:junit4] 2> 2006783 T4566 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1369904876005/awholynewcollection_0_shard1_replica2/data/tlog
[junit4:junit4] 2> 2006785 T4566 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2006785 T4566 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2006790 T4638 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard1_replica2] Registered new searcher Searcher@53fd221e main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2006795 T4566 oasc.ZkController.publish publishing core=awholynewcollection_0_shard1_replica2 state=down
[junit4:junit4] 2> 2006813 T4582 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2006827 T4582 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2006828 T4594 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 2006843 T4582 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2006888 T4581 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2006893 T4581 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2007018 T4594 oass.SolrIndexSearcher.<init> Opening Searcher@39ef2f53 main
[junit4:junit4] 2> 2007019 T4594 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1369904880436/awholynewcollection_0_shard3_replica3/data/tlog
[junit4:junit4] 2> 2007086 T4551 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2007101 T4551 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 2007108 T4594 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 2007109 T4594 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 2007113 T4541 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 2007115 T4541 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=5 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"5",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"awholynewcollection_0_shard1_replica2",
[junit4:junit4] 2> "collection":"awholynewcollection_0",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:50546_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:50546"}
[junit4:junit4] 2> 2007116 T4541 oasc.Overseer$ClusterStateUpdater.createCollection Create collection awholynewcollection_0 with numShards 5
[junit4:junit4] 2> 2007124 T4639 oasc.SolrCore.registerSearcher [awholynewcollection_0_shard3_replica3] Registered new searcher Searcher@39ef2f53 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 2007129 T4594 oasc.ZkController.publish publishing core=awholynewcollection_0_shard3_replica3 state=down
[junit4:junit4] 2> 2007131 T4615 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2007131 T4572 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2007131 T4540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2007133 T4600 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2007133 T4617 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2007134 T4545 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2007133 T4558 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2007136 T4577 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 2007167 T4586 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 2007182 T4551 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 2007199 T4536 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2007212 T4554 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 2> 2007273 T4552 oass.O
[...truncated too long message...]
on (5)
[junit4:junit4] 1> /solr/overseer_elect/election/89778085685100556-127.0.0.1:50552_-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89778085685100552-127.0.0.1:50546_-n_0000000002 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89778085685100550-127.0.0.1:50543_-n_0000000001 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89778085685100554-127.0.0.1:50549_-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89778085685100547-127.0.0.1:50539_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89778085685100547-127.0.0.1:50539_-n_0000000000"}
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=2BE5CE147F33D6FF -Dtests.slow=true -Dtests.locale=es_HN -Dtests.timezone=America/Merida -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 342s | CollectionsAPIDistributedZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([2BE5CE147F33D6FF:AA03400C086CB6C3]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
[junit4:junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
[junit4:junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
[junit4:junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:512)
[junit4:junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 2312069 T4518 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 342252 T4517 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> 2312846 T4600 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2312847 T4600 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> 2312847 T4600 oasc.LeaderElector$1.process WARN org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewcollection_2/leader_elect/shard3/election
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 2> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 2> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {timestamp=PostingsFormat(name=Lucene41WithOrds), range_facet_l=PostingsFormat(name=Lucene41WithOrds), id=PostingsFormat(name=Direct), range_facet_sl=PostingsFormat(name=Direct), a_t=PostingsFormat(name=Lucene41WithOrds), text=PostingsFormat(name=Memory doPackFST= false), range_facet_si=PostingsFormat(name=Lucene41WithOrds), _version_=PostingsFormat(name=Memory doPackFST= true), other_tl1=PostingsFormat(name=Memory doPackFST= true), multiDefault=PostingsFormat(name=Lucene41WithOrds), a_si=PostingsFormat(name=Memory doPackFST= true), intDefault=PostingsFormat(name=Memory doPackFST= true)}, docValues:{timestamp=DocValuesFormat(name=Disk)}, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_HN, timezone=America/Merida
[junit4:junit4] 2> NOTE: Mac OS X 10.8.3 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=2,free=48536704,total=227565568
[junit4:junit4] 2> NOTE: All tests run in this JVM: [RegexBoostProcessorTest, SolrIndexConfigTest, JsonLoaderTest, SyncSliceTest, TestCloudManagedSchema, QueryElevationComponentTest, NoCacheHeaderTest, ShardSplitTest, TestMultiCoreConfBootstrap, DirectSolrConnectionTest, PrimitiveFieldTypeTest, TestPropInjectDefaults, TestJoin, DistributedTermsComponentTest, TestConfig, TestSolrJ, TestFiltering, BinaryUpdateRequestHandlerTest, SolrPluginUtilsTest, TestMergePolicyConfig, TestLuceneMatchVersion, HighlighterConfigTest, TestSchemaSimilarityResource, TestSolrDeletionPolicy1, TestStressLucene, BadComponentTest, TestCollationField, TestDefaultSearchFieldResource, UnloadDistributedZkTest, DebugComponentTest, TestManagedSchema, TestQueryTypes, QueryEqualityTest, TestWordDelimiterFilterFactory, TestRTGBase, TestCoreDiscovery, IndexSchemaTest, SystemInfoHandlerTest, SignatureUpdateProcessorFactoryTest, SolrCoreCheckLockOnStartupTest, QueryParsingTest, DirectUpdateHandlerOptimizeTest, TestPartialUpdateDeduplication, TestReload, ClusterStateUpdateTest, DistanceFunctionTest, StandardRequestHandlerTest, TestDefaultSimilarityFactory, ChaosMonkeyShardSplitTest, TestFieldCollectionResource, TestQueryUtils, PreAnalyzedFieldTest, QueryResultKeyTest, TestSearchPerf, TestSurroundQueryParser, TestLazyCores, TestFoldingMultitermQuery, NumericFieldsTest, SuggesterWFSTTest, TestSchemaResource, CircularListTest, SchemaVersionSpecificBehaviorTest, DateMathParserTest, ClusterStateTest, ChaosMonkeySafeLeaderTest, SpellCheckComponentTest, AnalysisAfterCoreReloadTest, TestWriterPerf, SimplePostToolTest, DistributedSpellCheckComponentTest, WordBreakSolrSpellCheckerTest, TestCSVResponseWriter, OpenExchangeRatesOrgProviderTest, TestIndexSearcher, TestXIncludeConfig, TestGroupingSearch, PingRequestHandlerTest, TestValueSourceCache, DirectUpdateHandlerTest, TestIBSimilarityFactory, CurrencyFieldXmlFileTest, TestCoreContainer, TestDFRSimilarityFactory, ReturnFieldsTest, TestMaxScoreQueryParser, MoreLikeThisHandlerTest, SliceStateTest, FieldMutatingUpdateProcessorTest, TestAnalyzedSuggestions, RequiredFieldsTest, LeaderElectionTest, MinimalSchemaTest, StatsComponentTest, SolrCmdDistributorTest, FileBasedSpellCheckerTest, TestCopyFieldCollectionResource, BasicZkTest, TestLMJelinekMercerSimilarityFactory, SolrCoreTest, TestSchemaNameResource, TestSystemIdResolver, TestRemoteStreaming, URLClassifyProcessorTest, BasicDistributedZkTest, TestDistributedSearch, TestReplicationHandler, TestDocumentBuilder, TestSerializedLuceneMatchVersion, RAMDirectoryFactoryTest, UpdateParamsTest, CacheHeaderTest, TestCodecSupport, TestReversedWildcardFilterFactory, JSONWriterTest, SpellCheckCollatorTest, TestPseudoReturnFields, TestPHPSerializedResponseWriter, OutputWriterTest, BasicDistributedZk2Test, TestSweetSpotSimilarityFactory, SampleTest, TermVectorComponentDistributedTest, SoftAutoCommitTest, TestClassNameShortening, TestPluginEnable, FieldAnalysisRequestHandlerTest, TestLFUCache, TestExtendedDismaxParser, DocValuesTest, CoreContainerCoreInitFailuresTest, TestRecovery, TestIndexingPerformance, TestSolrQueryParserDefaultOperatorResource, AlternateDirectoryTest, DefaultValueUpdateProcessorTest, TestSolrDeletionPolicy2, IndexBasedSpellCheckerTest, TestShardHandlerFactory, UpdateRequestProcessorFactoryTest, UniqFieldsUpdateProcessorFactoryTest, TestSolrIndexConfig, SliceStateUpdateTest, OverseerTest, TestDocSet, TermVectorComponentTest, TestFuzzyAnalyzedSuggestions, CollectionsAPIDistributedZkTest]
[junit4:junit4] Completed in 343.31s, 1 test, 1 failure <<< FAILURES!
[...truncated 466 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:380: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:360: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1240: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:884: There were test failures: 295 suites, 1232 tests, 1 failure, 18 ignored (6 assumptions)
Total time: 120 minutes 3 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-MacOSX (64bit/jdk1.7.0) - Build # 520 -
Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-MacOSX/520/
Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
2 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:51419 returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:51419 returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([67B7C8B21887FED7:E65146AA6FD89EEB]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:208)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:136)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.ShardSplitTest.testDistribSearch
Error Message:
Server at http://127.0.0.1:51498/dhzk returned non ok status:500, message:Server Error
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:51498/dhzk returned non ok status:500, message:Server Error
at __randomizedtesting.SeedInfo.seed([67B7C8B21887FED7:E65146AA6FD89EEB]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:208)
at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:133)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
at sun.reflect.GeneratedMethodAccessor33.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9964 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4] 2> 4119188 T7343 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
[junit4:junit4] 2> 4119200 T7343 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
[junit4:junit4] 2> Creating dataDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./solrtest-ChaosMonkeyShardSplitTest-1369879070345
[junit4:junit4] 2> 4119203 T7343 oasc.ZkTestServer.run STARTING ZK TEST SERVER
[junit4:junit4] 2> 4119205 T7344 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
[junit4:junit4] 2> 4119308 T7343 oasc.ZkTestServer.run start zk server on port:51412
[junit4:junit4] 2> 4119312 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4119325 T7350 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@183a0c11 name:ZooKeeperConnection Watcher:127.0.0.1:51412 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4119326 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4119327 T7343 oascc.SolrZkClient.makePath makePath: /solr
[junit4:junit4] 2> 4119344 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4119356 T7352 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@54e67af9 name:ZooKeeperConnection Watcher:127.0.0.1:51412/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4119357 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4119357 T7343 oascc.SolrZkClient.makePath makePath: /collections/collection1
[junit4:junit4] 2> 4119369 T7343 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
[junit4:junit4] 2> 4119376 T7343 oascc.SolrZkClient.makePath makePath: /collections/control_collection
[junit4:junit4] 2> 4119385 T7343 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
[junit4:junit4] 2> 4119402 T7343 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 4119403 T7343 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 2> 4119422 T7343 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4:junit4] 2> 4119424 T7343 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
[junit4:junit4] 2> 4119435 T7343 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 2> 4119436 T7343 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 2> 4119446 T7343 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 2> 4119448 T7343 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
[junit4:junit4] 2> 4119457 T7343 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 2> 4119459 T7343 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
[junit4:junit4] 2> 4119468 T7343 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 4119469 T7343 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 2> 4119478 T7343 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 4119479 T7343 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 2> 4119488 T7343 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 4119490 T7343 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 2> 4119505 T7343 oasc.AbstractZkTestCase.putConfig put /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 2> 4119507 T7343 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 2> 4120332 T7343 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 4120361 T7343 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51415
[junit4:junit4] 2> 4120363 T7343 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 4120364 T7343 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 4120365 T7343 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369879070666
[junit4:junit4] 2> 4120366 T7343 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369879070666/solr.xml
[junit4:junit4] 2> 4120367 T7343 oasc.CoreContainer.<init> New CoreContainer 1654400925
[junit4:junit4] 2> 4120368 T7343 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369879070666/'
[junit4:junit4] 2> 4120369 T7343 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369879070666/'
[junit4:junit4] 2> 4120666 T3 oasu.ConcurrentLRUCache.finalize ERROR ConcurrentLRUCache was not destroyed prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!
[junit4:junit4] 2> 4120706 T7343 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 4120707 T7343 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 4120708 T7343 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 4120709 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 4120710 T7343 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 4120711 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 4120711 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 4120712 T7343 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 4120713 T7343 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 4120714 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 4120755 T7343 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 4120756 T7343 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51412/solr
[junit4:junit4] 2> 4120757 T7343 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 4120760 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4120769 T7363 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61a5662 name:ZooKeeperConnection Watcher:127.0.0.1:51412 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4120770 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4120775 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 4120805 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4120812 T7365 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@73b0a30e name:ZooKeeperConnection Watcher:127.0.0.1:51412/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4120816 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4120822 T7343 oascc.SolrZkClient.makePath makePath: /overseer/queue
[junit4:junit4] 2> 4120833 T7343 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
[junit4:junit4] 2> 4120844 T7343 oascc.SolrZkClient.makePath makePath: /live_nodes
[junit4:junit4] 2> 4120850 T7343 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51415_
[junit4:junit4] 2> 4120853 T7343 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51415_
[junit4:junit4] 2> 4120864 T7343 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
[junit4:junit4] 2> 4120890 T7343 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
[junit4:junit4] 2> 4120899 T7343 oasc.Overseer.start Overseer (id=89776394755178499-127.0.0.1:51415_-n_0000000000) starting
[junit4:junit4] 2> 4120913 T7343 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
[junit4:junit4] 2> 4120928 T7367 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
[junit4:junit4] 2> 4120935 T7343 oascc.SolrZkClient.makePath makePath: /clusterstate.json
[junit4:junit4] 2> 4120949 T7343 oascc.SolrZkClient.makePath makePath: /aliases.json
[junit4:junit4] 2> 4120956 T7343 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4120966 T7366 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
[junit4:junit4] 2> 4120982 T7368 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369879070666/collection1
[junit4:junit4] 2> 4120982 T7368 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
[junit4:junit4] 2> 4120984 T7368 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4120984 T7368 oasc.ZkController.readConfigName Load collection config from:/collections/control_collection
[junit4:junit4] 2> 4120988 T7368 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369879070666/collection1/'
[junit4:junit4] 2> 4120990 T7368 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369879070666/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4120991 T7368 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369879070666/collection1/lib/README' to classloader
[junit4:junit4] 2> 4121098 T7368 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 4121234 T7368 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4121238 T7368 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 4121246 T7368 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 4122415 T7368 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 4122421 T7368 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 4122426 T7368 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4122438 T7368 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4122520 T7368 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 4122521 T7368 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1369879070666/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/control/data/
[junit4:junit4] 2> 4122522 T7368 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d16428d
[junit4:junit4] 2> 4122523 T7368 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 4122524 T7368 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/control/data
[junit4:junit4] 2> 4122525 T7368 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/control/data/index/
[junit4:junit4] 2> 4122525 T7368 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 4122530 T7368 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/control/data/index
[junit4:junit4] 2> 4122544 T7368 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@71bb44a3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@404f58f6),segFN=segments_1,generation=1}
[junit4:junit4] 2> 4122545 T7368 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 4122552 T7368 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 4122553 T7368 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 4122555 T7368 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 4122557 T7368 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 4122558 T7368 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 4122559 T7368 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 4122560 T7368 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 4122563 T7368 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 4122564 T7368 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 4122591 T7368 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 4122610 T7368 oass.SolrIndexSearcher.<init> Opening Searcher@6a0921ad main
[junit4:junit4] 2> 4122611 T7368 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/control/data/tlog
[junit4:junit4] 2> 4122615 T7368 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 4122615 T7368 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 4122624 T7369 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a0921ad main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 4122626 T7368 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 4122627 T7368 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4123985 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4123991 T7366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51415_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51415"}
[junit4:junit4] 2> 4123993 T7366 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with numShards 1
[junit4:junit4] 2> 4123993 T7366 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 4124011 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 4124632 T7368 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 4124632 T7368 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51415 collection:control_collection shard:shard1
[junit4:junit4] 2> 4124634 T7368 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 2> 4124653 T7368 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 4124665 T7368 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 4124666 T7368 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 4124666 T7368 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51415/collection1/
[junit4:junit4] 2> 4124667 T7368 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 4124667 T7368 oasc.SyncStrategy.syncToMe http://127.0.0.1:51415/collection1/ has no replicas
[junit4:junit4] 2> 4124668 T7368 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51415/collection1/
[junit4:junit4] 2> 4124668 T7368 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 2> 4125524 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4125560 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 4125576 T7368 oasc.ZkController.register We are http://127.0.0.1:51415/collection1/ and leader is http://127.0.0.1:51415/collection1/
[junit4:junit4] 2> 4125577 T7368 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51415
[junit4:junit4] 2> 4125577 T7368 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 4125578 T7368 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 4125578 T7368 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4125582 T7368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4125589 T7343 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 4125594 T7343 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 4125596 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 4125630 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:
[junit4:junit4] 2> 4125635 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4125643 T7372 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57523e7f name:ZooKeeperConnection Watcher:127.0.0.1:51412/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4125644 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4125648 T7343 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4125657 T7343 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 2> 4126371 T7343 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 4126383 T7343 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51419
[junit4:junit4] 2> 4126388 T7343 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 4126389 T7343 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 4126389 T7343 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369879076804
[junit4:junit4] 2> 4126391 T7343 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369879076804/solr.xml
[junit4:junit4] 2> 4126392 T7343 oasc.CoreContainer.<init> New CoreContainer 999208318
[junit4:junit4] 2> 4126393 T7343 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369879076804/'
[junit4:junit4] 2> 4126393 T7343 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369879076804/'
[junit4:junit4] 2> 4126676 T7343 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 4126677 T7343 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 4126677 T7343 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 4126678 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 4126679 T7343 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 4126679 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 4126680 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 4126681 T7343 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 4126681 T7343 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 4126682 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 4126711 T7343 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 4126712 T7343 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51412/solr
[junit4:junit4] 2> 4126713 T7343 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 4126722 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4126728 T7383 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@738f0da2 name:ZooKeeperConnection Watcher:127.0.0.1:51412 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4126730 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4126734 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 4126756 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4126761 T7385 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@674571c1 name:ZooKeeperConnection Watcher:127.0.0.1:51412/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4126762 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4126774 T7343 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4127078 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4127080 T7366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51415__collection1",
[junit4:junit4] 2> "numShards":"1",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"control_collection",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51415_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51415"}
[junit4:junit4] 2> 4127092 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 4127093 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 4127100 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 2> 4127798 T7343 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51419_
[junit4:junit4] 2> 4127800 T7343 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51419_
[junit4:junit4] 2> 4127816 T7372 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4127816 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4127817 T7385 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4127817 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4127818 T7365 oascc.ZkStateReader$3.process Updating live nodes... (2)
[junit4:junit4] 2> 4127818 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4127843 T7386 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369879076804/collection1
[junit4:junit4] 2> 4127844 T7386 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 4127845 T7386 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4127846 T7386 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 4127852 T7386 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369879076804/collection1/'
[junit4:junit4] 2> 4127858 T7386 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369879076804/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4127862 T7386 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369879076804/collection1/lib/README' to classloader
[junit4:junit4] 2> 4127953 T7386 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 4128136 T7386 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4128140 T7386 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 4128146 T7386 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 4129131 T7386 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 4129136 T7386 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 4129139 T7386 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4129149 T7386 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4129220 T7386 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 4129221 T7386 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1369879076804/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty1/
[junit4:junit4] 2> 4129221 T7386 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d16428d
[junit4:junit4] 2> 4129222 T7386 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 4129226 T7386 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty1
[junit4:junit4] 2> 4129229 T7386 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty1/index/
[junit4:junit4] 2> 4129230 T7386 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 4129233 T7386 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty1/index
[junit4:junit4] 2> 4129238 T7386 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@42d48dad lockFactory=org.apache.lucene.store.NativeFSLockFactory@4e86a408),segFN=segments_1,generation=1}
[junit4:junit4] 2> 4129238 T7386 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 4129246 T7386 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 4129247 T7386 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 4129248 T7386 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 4129249 T7386 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 4129250 T7386 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 4129251 T7386 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 4129251 T7386 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 4129253 T7386 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 4129254 T7386 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 4129275 T7386 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 4129292 T7386 oass.SolrIndexSearcher.<init> Opening Searcher@c9264b5 main
[junit4:junit4] 2> 4129293 T7386 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty1/tlog
[junit4:junit4] 2> 4129295 T7386 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 4129296 T7386 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 4129306 T7387 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@c9264b5 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 4129312 T7386 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 4129313 T7386 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4130115 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4130120 T7366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51419_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51419"}
[junit4:junit4] 2> 4130120 T7366 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with numShards 2
[junit4:junit4] 2> 4130121 T7366 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 4130134 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4130135 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4130135 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4130328 T7386 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 4130328 T7386 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51419 collection:collection1 shard:shard1
[junit4:junit4] 2> 4130330 T7386 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 2> 4130354 T7386 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 4130363 T7386 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 4130364 T7386 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 4130364 T7386 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51419/collection1/
[junit4:junit4] 2> 4130365 T7386 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 4130365 T7386 oasc.SyncStrategy.syncToMe http://127.0.0.1:51419/collection1/ has no replicas
[junit4:junit4] 2> 4130366 T7386 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51419/collection1/
[junit4:junit4] 2> 4130368 T7386 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 2> 4131656 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4131704 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4131704 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4131705 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4131745 T7386 oasc.ZkController.register We are http://127.0.0.1:51419/collection1/ and leader is http://127.0.0.1:51419/collection1/
[junit4:junit4] 2> 4131746 T7386 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51419
[junit4:junit4] 2> 4131746 T7386 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 4131747 T7386 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 4131747 T7386 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4131767 T7386 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4131771 T7343 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 4131771 T7343 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 4131772 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 4132713 T7343 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 4132738 T7343 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51422
[junit4:junit4] 2> 4132739 T7343 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 4132740 T7343 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 4132741 T7343 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369879082961
[junit4:junit4] 2> 4132770 T7343 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369879082961/solr.xml
[junit4:junit4] 2> 4132771 T7343 oasc.CoreContainer.<init> New CoreContainer 1685865596
[junit4:junit4] 2> 4132773 T7343 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369879082961/'
[junit4:junit4] 2> 4132774 T7343 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369879082961/'
[junit4:junit4] 2> 4133066 T7343 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 4133067 T7343 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 4133068 T7343 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 4133069 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 4133070 T7343 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 4133070 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 4133071 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 4133072 T7343 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 4133072 T7343 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 4133073 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 4133111 T7343 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 4133112 T7343 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51412/solr
[junit4:junit4] 2> 4133113 T7343 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 4133115 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4133123 T7399 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cbb7687 name:ZooKeeperConnection Watcher:127.0.0.1:51412 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4133126 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4133132 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 4133169 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4133183 T7401 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7d8aa4d name:ZooKeeperConnection Watcher:127.0.0.1:51412/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4133184 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4133230 T7343 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4133232 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4133240 T7366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51419__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51419_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51419"}
[junit4:junit4] 2> 4133265 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4133266 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4133269 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4133270 T7401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 2> 4134265 T7343 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51422_
[junit4:junit4] 2> 4134269 T7343 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51422_
[junit4:junit4] 2> 4134279 T7365 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 4134279 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4134280 T7401 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 4134280 T7401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4134281 T7372 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 4134282 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4134281 T7385 oascc.ZkStateReader$3.process Updating live nodes... (3)
[junit4:junit4] 2> 4134283 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4134312 T7402 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369879082961/collection1
[junit4:junit4] 2> 4134313 T7402 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 4134320 T7402 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4134321 T7402 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 4134329 T7402 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369879082961/collection1/'
[junit4:junit4] 2> 4134331 T7402 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369879082961/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4134331 T7402 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369879082961/collection1/lib/README' to classloader
[junit4:junit4] 2> 4134463 T7402 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 4134610 T7402 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4134622 T7402 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 4134631 T7402 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 4136073 T7402 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 4136079 T7402 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 4136082 T7402 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4136102 T7402 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4136296 T7402 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 4136298 T7402 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1369879082961/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty2/
[junit4:junit4] 2> 4136298 T7402 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d16428d
[junit4:junit4] 2> 4136299 T7402 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 4136307 T7402 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty2
[junit4:junit4] 2> 4136319 T7402 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty2/index/
[junit4:junit4] 2> 4136321 T7402 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 4136323 T7402 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty2/index
[junit4:junit4] 2> 4136333 T7402 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4d937b37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@47bb53a4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 4136334 T7402 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 4136341 T7402 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 4136342 T7402 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 4136343 T7402 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 4136344 T7402 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 4136345 T7402 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 4136345 T7402 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 4136346 T7402 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 4136348 T7402 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 4136349 T7402 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 4136371 T7402 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 4136392 T7402 oass.SolrIndexSearcher.<init> Opening Searcher@183890cd main
[junit4:junit4] 2> 4136393 T7402 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty2/tlog
[junit4:junit4] 2> 4136397 T7402 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 4136397 T7402 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 4136412 T7403 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@183890cd main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 4136416 T7402 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 4136416 T7402 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4137813 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4137820 T7366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51422_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51422"}
[junit4:junit4] 2> 4137820 T7366 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 4137821 T7366 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 4137834 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4137834 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4137835 T7401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4137834 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4138439 T7402 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 4138439 T7402 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51422 collection:collection1 shard:shard2
[junit4:junit4] 2> 4138442 T7402 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 2> 4138474 T7402 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process.
[junit4:junit4] 2> 4138480 T7402 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
[junit4:junit4] 2> 4138480 T7402 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
[junit4:junit4] 2> 4138481 T7402 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:51422/collection1/
[junit4:junit4] 2> 4138481 T7402 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
[junit4:junit4] 2> 4138482 T7402 oasc.SyncStrategy.syncToMe http://127.0.0.1:51422/collection1/ has no replicas
[junit4:junit4] 2> 4138482 T7402 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:51422/collection1/
[junit4:junit4] 2> 4138482 T7402 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 2> 4139358 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4139378 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4139379 T7401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4139381 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4139382 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4139422 T7402 oasc.ZkController.register We are http://127.0.0.1:51422/collection1/ and leader is http://127.0.0.1:51422/collection1/
[junit4:junit4] 2> 4139424 T7402 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51422
[junit4:junit4] 2> 4139424 T7402 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
[junit4:junit4] 2> 4139425 T7402 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 4139426 T7402 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4139432 T7402 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4139435 T7343 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 4139436 T7343 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 4139437 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 4140221 T7343 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 4140243 T7343 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51425
[junit4:junit4] 2> 4140254 T7343 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 4140254 T7343 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 4140255 T7343 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369879090622
[junit4:junit4] 2> 4140257 T7343 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369879090622/solr.xml
[junit4:junit4] 2> 4140257 T7343 oasc.CoreContainer.<init> New CoreContainer 528189205
[junit4:junit4] 2> 4140258 T7343 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369879090622/'
[junit4:junit4] 2> 4140259 T7343 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369879090622/'
[junit4:junit4] 2> 4140566 T7343 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 4140567 T7343 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 4140568 T7343 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 4140568 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 4140568 T7343 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 4140569 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 4140569 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 4140570 T7343 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 4140570 T7343 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 4140571 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 4140628 T7343 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 4140629 T7343 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51412/solr
[junit4:junit4] 2> 4140638 T7343 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 4140643 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4140659 T7415 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3a04eb59 name:ZooKeeperConnection Watcher:127.0.0.1:51412 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4140660 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4140673 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 4140722 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4140728 T7417 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@378eb4f3 name:ZooKeeperConnection Watcher:127.0.0.1:51412/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4140731 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4140758 T7343 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4140915 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4140917 T7366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51422__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51422_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51422"}
[junit4:junit4] 2> 4140936 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4140937 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4140937 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4140939 T7401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4140940 T7417 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4141777 T7343 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51425_
[junit4:junit4] 2> 4141782 T7343 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51425_
[junit4:junit4] 2> 4141790 T7417 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 2> 4141793 T7365 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 4141793 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4141794 T7401 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 4141794 T7372 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 4141796 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4141794 T7385 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 4141796 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4141795 T7401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4141804 T7417 oascc.ZkStateReader$3.process Updating live nodes... (4)
[junit4:junit4] 2> 4141824 T7418 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369879090622/collection1
[junit4:junit4] 2> 4141825 T7418 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 4141829 T7418 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4141829 T7418 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 4141835 T7418 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369879090622/collection1/'
[junit4:junit4] 2> 4141838 T7418 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369879090622/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4141839 T7418 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369879090622/collection1/lib/README' to classloader
[junit4:junit4] 2> 4141946 T7418 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 4142064 T7418 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4142075 T7418 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 4142084 T7418 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> 4143238 T7418 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 4143259 T7418 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 4143264 T7418 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4143291 T7418 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4143401 T7418 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 4143403 T7418 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1369879090622/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty3/
[junit4:junit4] 2> 4143404 T7418 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d16428d
[junit4:junit4] 2> 4143405 T7418 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 4143406 T7418 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty3
[junit4:junit4] 2> 4143406 T7418 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty3/index/
[junit4:junit4] 2> 4143407 T7418 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 4143411 T7418 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty3/index
[junit4:junit4] 2> 4143416 T7418 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b77016b lockFactory=org.apache.lucene.store.NativeFSLockFactory@7511c1fd),segFN=segments_1,generation=1}
[junit4:junit4] 2> 4143416 T7418 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 4143421 T7418 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 4143421 T7418 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 4143422 T7418 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 4143423 T7418 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 4143424 T7418 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 4143425 T7418 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 4143426 T7418 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 4143428 T7418 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 4143429 T7418 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 4143449 T7418 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 4143465 T7418 oass.SolrIndexSearcher.<init> Opening Searcher@15a3f1f4 main
[junit4:junit4] 2> 4143466 T7418 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty3/tlog
[junit4:junit4] 2> 4143467 T7418 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 4143468 T7418 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 4143476 T7419 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@15a3f1f4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 4143480 T7418 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 4143480 T7418 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4143976 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4143978 T7366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51425_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51425"}
[junit4:junit4] 2> 4143979 T7366 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 4143979 T7366 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard1
[junit4:junit4] 2> 4143992 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4143995 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4143999 T7417 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4143999 T7401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4143999 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4144485 T7418 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 4144485 T7418 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51425 collection:collection1 shard:shard1
[junit4:junit4] 2> 4144493 T7418 oasc.ZkController.register We are http://127.0.0.1:51425/collection1/ and leader is http://127.0.0.1:51419/collection1/
[junit4:junit4] 2> 4144493 T7418 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51425
[junit4:junit4] 2> 4144494 T7418 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 4144494 T7418 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1948 name=collection1 org.apache.solr.core.SolrCore@5ddac2e0 url=http://127.0.0.1:51425/collection1 node=127.0.0.1:51425_ C1948_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51425_, base_url=http://127.0.0.1:51425}
[junit4:junit4] 2> 4144495 T7420 C1948 P51425 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 4144499 T7420 C1948 P51425 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 4144499 T7420 C1948 P51425 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 4144500 T7420 C1948 P51425 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4144501 T7418 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4144503 T7420 C1948 P51425 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 4144533 T7343 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 4144534 T7343 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 4144536 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 4144541 T7381 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:51425__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 4145390 T7343 oejs.Server.doStart jetty-8.1.10.v20130312
[junit4:junit4] 2> 4145424 T7343 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:51429
[junit4:junit4] 2> 4145430 T7343 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
[junit4:junit4] 2> 4145431 T7343 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 2> 4145432 T7343 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369879095705
[junit4:junit4] 2> 4145432 T7343 oasc.CoreContainer$Initializer.initialize looking for solr config file: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369879095705/solr.xml
[junit4:junit4] 2> 4145433 T7343 oasc.CoreContainer.<init> New CoreContainer 1406620229
[junit4:junit4] 2> 4145434 T7343 oasc.CoreContainer.load Loading CoreContainer using Solr Home: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369879095705/'
[junit4:junit4] 2> 4145435 T7343 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369879095705/'
[junit4:junit4] 2> 4145511 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4145516 T7366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51425__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51425_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51425"}
[junit4:junit4] 2> 4145551 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4145551 T7401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4145552 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4145552 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4145552 T7417 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4145750 T7343 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 120000
[junit4:junit4] 2> 4145751 T7343 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
[junit4:junit4] 2> 4145752 T7343 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
[junit4:junit4] 2> 4145752 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
[junit4:junit4] 2> 4145753 T7343 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
[junit4:junit4] 2> 4145754 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
[junit4:junit4] 2> 4145755 T7343 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
[junit4:junit4] 2> 4145756 T7343 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
[junit4:junit4] 2> 4145759 T7343 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
[junit4:junit4] 2> 4145764 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 2> 4145802 T7343 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener
[junit4:junit4] 2> 4145813 T7343 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:51412/solr
[junit4:junit4] 2> 4145815 T7343 oasc.ZkController.checkChrootPath zkHost includes chroot
[junit4:junit4] 2> 4145820 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4145827 T7432 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@245a5444 name:ZooKeeperConnection Watcher:127.0.0.1:51412 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4145835 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4145847 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 2> 4145886 T7343 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
[junit4:junit4] 2> 4145894 T7434 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@773677f name:ZooKeeperConnection Watcher:127.0.0.1:51412/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 2> 4145898 T7343 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
[junit4:junit4] 2> 4145929 T7343 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper...
[junit4:junit4] 2> 4146545 T7381 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:51425__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 4146547 T7381 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:51425__collection1&state=recovering&nodeName=127.0.0.1:51425_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2006
[junit4:junit4] 2> 4146977 T7343 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51429_
[junit4:junit4] 2> 4147102 T7343 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:51429_
[junit4:junit4] 2> 4147216 T7417 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 2> 4147218 T7365 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 4147218 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4147219 T7385 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 4147219 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4147224 T7434 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 4147225 T7372 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 4147226 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4147226 T7401 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 4147227 T7401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4147237 T7417 oascc.ZkStateReader$3.process Updating live nodes... (5)
[junit4:junit4] 2> 4147278 T7435 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369879095705/collection1
[junit4:junit4] 2> 4147278 T7435 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
[junit4:junit4] 2> 4147280 T7435 oasc.ZkController.createCollectionZkNode Collection zkNode exists
[junit4:junit4] 2> 4147281 T7435 oasc.ZkController.readConfigName Load collection config from:/collections/collection1
[junit4:junit4] 2> 4147286 T7435 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369879095705/collection1/'
[junit4:junit4] 2> 4147289 T7435 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369879095705/collection1/lib/classes/' to classloader
[junit4:junit4] 2> 4147289 T7435 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369879095705/collection1/lib/README' to classloader
[junit4:junit4] 2> 4147432 T7435 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 2> 4147628 T7435 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 2> 4147644 T7435 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
[junit4:junit4] 2> 4147656 T7435 oass.IndexSchema.readSchema [collection1] Schema name=test
[junit4:junit4] 2> ASYNC NEW_CORE C1949 name=collection1 org.apache.solr.core.SolrCore@5ddac2e0 url=http://127.0.0.1:51425/collection1 node=127.0.0.1:51425_ C1949_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51425_, base_url=http://127.0.0.1:51425}
[junit4:junit4] 2> 4148551 T7420 C1949 P51425 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51419/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 4148552 T7420 C1949 P51425 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51425 START replicas=[http://127.0.0.1:51419/collection1/] nUpdates=100
[junit4:junit4] 2> 4148553 T7420 C1949 P51425 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 4148559 T7420 C1949 P51425 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 4148560 T7420 C1949 P51425 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 4148560 T7420 C1949 P51425 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 4148560 T7420 C1949 P51425 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 4148561 T7420 C1949 P51425 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51419/collection1/. core=collection1
[junit4:junit4] 2> 4148561 T7420 C1949 P51425 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1950 name=collection1 org.apache.solr.core.SolrCore@77ce6786 url=http://127.0.0.1:51419/collection1 node=127.0.0.1:51419_ C1950_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51419_, base_url=http://127.0.0.1:51419, leader=true}
[junit4:junit4] 2> 4148645 T7377 C1950 P51419 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=14
[junit4:junit4] 2> 4148651 T7379 C1950 P51419 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 4148673 T7379 C1950 P51419 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@42d48dad lockFactory=org.apache.lucene.store.NativeFSLockFactory@4e86a408),segFN=segments_1,generation=1}
[junit4:junit4] 2> 4148686 T7379 C1950 P51419 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 4148691 T7379 C1950 P51419 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@42d48dad lockFactory=org.apache.lucene.store.NativeFSLockFactory@4e86a408),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@42d48dad lockFactory=org.apache.lucene.store.NativeFSLockFactory@4e86a408),segFN=segments_2,generation=2}
[junit4:junit4] 2> 4148692 T7379 C1950 P51419 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 4148693 T7379 C1950 P51419 oass.SolrIndexSearcher.<init> Opening Searcher@61a615c6 realtime
[junit4:junit4] 2> 4148693 T7379 C1950 P51419 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 4148694 T7379 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 43
[junit4:junit4] 2> 4148697 T7420 C1949 P51425 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 4148697 T7420 C1949 P51425 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 4148709 T7376 C1950 P51419 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 4148710 T7376 C1950 P51419 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=9
[junit4:junit4] 2> 4148712 T7420 C1949 P51425 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 4148712 T7420 C1949 P51425 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 4148713 T7420 C1949 P51425 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 4148727 T7381 C1950 P51419 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=11
[junit4:junit4] 2> 4148729 T7420 C1949 P51425 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 4148749 T7420 C1949 P51425 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty3/index.20130529185819872
[junit4:junit4] 2> 4148750 T7420 C1949 P51425 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@19ac398b lockFactory=org.apache.lucene.store.NativeFSLockFactory@4f692ce0) fullCopy=false
[junit4:junit4] 2> 4148761 T7378 C1950 P51419 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=0
[junit4:junit4] 2> 4148782 T7420 C1949 P51425 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 4148803 T7420 C1949 P51425 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 4148804 T7420 C1949 P51425 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 4148815 T7420 C1949 P51425 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b77016b lockFactory=org.apache.lucene.store.NativeFSLockFactory@7511c1fd),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b77016b lockFactory=org.apache.lucene.store.NativeFSLockFactory@7511c1fd),segFN=segments_2,generation=2}
[junit4:junit4] 2> 4148820 T7420 C1949 P51425 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 4148826 T7420 C1949 P51425 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 4148827 T7420 C1949 P51425 oass.SolrIndexSearcher.<init> Opening Searcher@1412b559 main
[junit4:junit4] 2> 4148830 T7419 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1412b559 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 4148832 T7420 C1949 P51425 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty3/index.20130529185819872 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty3/index.20130529185819872;done=true>>]
[junit4:junit4] 2> 4148832 T7420 C1949 P51425 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty3/index.20130529185819872
[junit4:junit4] 2> 4148842 T7420 C1949 P51425 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty3/index.20130529185819872
[junit4:junit4] 2> 4148842 T7420 C1949 P51425 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 4148843 T7420 C1949 P51425 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 4148843 T7420 C1949 P51425 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 4148843 T7420 C1949 P51425 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4148846 T7420 C1949 P51425 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 4149337 T7435 oass.IndexSchema.readSchema default search field in schema is text
[junit4:junit4] 2> 4149368 T7435 oass.IndexSchema.readSchema unique key field: id
[junit4:junit4] 2> 4149371 T7435 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4149388 T7435 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
[junit4:junit4] 2> 4149527 T7435 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 4149528 T7435 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1369879095705/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty4/
[junit4:junit4] 2> 4149528 T7435 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@d16428d
[junit4:junit4] 2> 4149529 T7435 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 2> 4149530 T7435 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty4
[junit4:junit4] 2> 4149530 T7435 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty4/index/
[junit4:junit4] 2> 4149531 T7435 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 2> 4149533 T7435 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty4/index
[junit4:junit4] 2> 4149539 T7435 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@474e8210 lockFactory=org.apache.lucene.store.NativeFSLockFactory@604033a4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 4149539 T7435 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 4149552 T7435 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 2> 4149553 T7435 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 2> 4149554 T7435 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 2> 4149558 T7435 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
[junit4:junit4] 2> 4149559 T7435 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
[junit4:junit4] 2> 4149559 T7435 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 2> 4149565 T7435 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
[junit4:junit4] 2> 4149567 T7435 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
[junit4:junit4] 2> 4149568 T7435 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 2> 4149590 T7435 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
[junit4:junit4] 2> 4149609 T7435 oass.SolrIndexSearcher.<init> Opening Searcher@6b8805c0 main
[junit4:junit4] 2> 4149610 T7435 oasu.UpdateHandler.clearLog Clearing tlog files, tlogDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty4/tlog
[junit4:junit4] 2> 4149612 T7435 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4:junit4] 2> 4149613 T7435 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4:junit4] 2> 4149626 T7438 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6b8805c0 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 2> 4149630 T7435 oasc.ZkController.publish publishing core=collection1 state=down
[junit4:junit4] 2> 4149631 T7435 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4150143 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4150145 T7366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51425__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard1",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51425_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51425"}
[junit4:junit4] 2> 4150157 T7366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":null,
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":null,
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"down",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51429_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51429"}
[junit4:junit4] 2> 4150158 T7366 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
[junit4:junit4] 2> 4150158 T7366 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard=shard2
[junit4:junit4] 2> 4150172 T7417 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4150173 T7434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4150173 T7401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4150173 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4150173 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4150173 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4150634 T7435 oasc.CoreContainer.registerCore registering core: collection1
[junit4:junit4] 2> 4150635 T7435 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:51429 collection:collection1 shard:shard2
[junit4:junit4] 2> 4150645 T7435 oasc.ZkController.register We are http://127.0.0.1:51429/collection1/ and leader is http://127.0.0.1:51422/collection1/
[junit4:junit4] 2> 4150645 T7435 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:51429
[junit4:junit4] 2> 4150646 T7435 oasc.ZkController.checkRecovery Core needs to recover:collection1
[junit4:junit4] 2> 4150646 T7435 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
[junit4:junit4] 2> ASYNC NEW_CORE C1951 name=collection1 org.apache.solr.core.SolrCore@4dbcd1f9 url=http://127.0.0.1:51429/collection1 node=127.0.0.1:51429_ C1951_STATE=coll:collection1 core:collection1 props:{state=down, core=collection1, collection=collection1, node_name=127.0.0.1:51429_, base_url=http://127.0.0.1:51429}
[junit4:junit4] 2> 4150648 T7439 C1951 P51429 oasc.RecoveryStrategy.run Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 2> 4150649 T7435 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4150651 T7439 C1951 P51429 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
[junit4:junit4] 2> 4150653 T7439 C1951 P51429 oasc.ZkController.publish publishing core=collection1 state=recovering
[junit4:junit4] 2> 4150653 T7439 C1951 P51429 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4150660 T7439 C1951 P51429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 4150661 T7343 oass.SolrDispatchFilter.init user.dir=/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0
[junit4:junit4] 2> 4150662 T7343 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
[junit4:junit4] 2> 4150663 T7343 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> 4150699 T7343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4150704 T7391 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: 127.0.0.1:51429__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 2> 4150713 T7343 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 2> 4150717 T7343 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 2> 4150719 T7343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4151696 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4151698 T7366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51429__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"recovering",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51429_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51429"}
[junit4:junit4] 2> 4151711 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4151711 T7417 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4151717 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4151718 T7401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4151719 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4151719 T7434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4151734 T7343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4152707 T7391 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: 127.0.0.1:51429__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 2> 4152708 T7391 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={coreNodeName=127.0.0.1:51429__collection1&state=recovering&nodeName=127.0.0.1:51429_&action=PREPRECOVERY&checkLive=true&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=2004
[junit4:junit4] 2> 4152739 T7343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4153744 T7343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> C1951_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=recovering, core=collection1, collection=collection1, node_name=127.0.0.1:51429_, base_url=http://127.0.0.1:51429}
[junit4:junit4] 2> 4154710 T7439 C1951 P51429 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:51422/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 2> 4154711 T7439 C1951 P51429 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:51429 START replicas=[http://127.0.0.1:51422/collection1/] nUpdates=100
[junit4:junit4] 2> 4154712 T7439 C1951 P51429 oasu.PeerSync.sync WARN no frame of reference to tell of we've missed updates
[junit4:junit4] 2> 4154712 T7439 C1951 P51429 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 2> 4154712 T7439 C1951 P51429 oasc.RecoveryStrategy.doRecovery Starting Replication Recovery. core=collection1
[junit4:junit4] 2> 4154713 T7439 C1951 P51429 oasc.RecoveryStrategy.doRecovery Begin buffering updates. core=collection1
[junit4:junit4] 2> 4154713 T7439 C1951 P51429 oasu.UpdateLog.bufferUpdates Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 2> 4154713 T7439 C1951 P51429 oasc.RecoveryStrategy.replicate Attempting to replicate from http://127.0.0.1:51422/collection1/. core=collection1
[junit4:junit4] 2> 4154714 T7439 C1951 P51429 oascsi.HttpClientUtil.createClient Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 2> ASYNC NEW_CORE C1952 name=collection1 org.apache.solr.core.SolrCore@3d9d4783 url=http://127.0.0.1:51422/collection1 node=127.0.0.1:51422_ C1952_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51422_, base_url=http://127.0.0.1:51422, leader=true}
[junit4:junit4] 2> 4154750 T7391 C1952 P51422 oasc.SolrCore.execute [collection1] webapp= path=/get params={getVersions=100&distrib=false&wt=javabin&qt=/get&version=2} status=0 QTime=16
[junit4:junit4] 2> 4154751 T7343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4154760 T7393 C1952 P51422 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 4154765 T7393 C1952 P51422 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4d937b37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@47bb53a4),segFN=segments_1,generation=1}
[junit4:junit4] 2> 4154774 T7393 C1952 P51422 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 4154776 T7393 C1952 P51422 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4d937b37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@47bb53a4),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4d937b37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@47bb53a4),segFN=segments_2,generation=2}
[junit4:junit4] 2> 4154777 T7393 C1952 P51422 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 4154778 T7393 C1952 P51422 oass.SolrIndexSearcher.<init> Opening Searcher@7b8d6376 realtime
[junit4:junit4] 2> 4154779 T7393 C1952 P51422 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 4154779 T7393 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&openSearcher=false&commit=true&wt=javabin&commit_end_point=true&version=2&softCommit=false} {commit=} 0 19
[junit4:junit4] 2> 4154782 T7439 C1951 P51429 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 4154782 T7439 C1951 P51429 oash.SnapPuller.<init> No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 2> 4154792 T7395 C1952 P51422 oash.ReplicationHandler.inform Commits will be reserved for 10000
[junit4:junit4] 2> 4154793 T7395 C1952 P51422 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=2
[junit4:junit4] 2> 4154794 T7439 C1951 P51429 oash.SnapPuller.fetchLatestIndex Master's generation: 2
[junit4:junit4] 2> 4154795 T7439 C1951 P51429 oash.SnapPuller.fetchLatestIndex Slave's generation: 1
[junit4:junit4] 2> 4154795 T7439 C1951 P51429 oash.SnapPuller.fetchLatestIndex Starting replication process
[junit4:junit4] 2> 4154800 T7396 C1952 P51422 oasc.SolrCore.execute [collection1] webapp= path=/replication params={command=filelist&qt=/replication&wt=javabin&generation=2&version=2} status=0 QTime=1
[junit4:junit4] 2> 4154802 T7439 C1951 P51429 oash.SnapPuller.fetchLatestIndex Number of files in latest index in master: 1
[junit4:junit4] 2> 4154806 T7439 C1951 P51429 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty4/index.20130529185825946
[junit4:junit4] 2> 4154806 T7439 C1951 P51429 oash.SnapPuller.fetchLatestIndex Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@67955da0 lockFactory=org.apache.lucene.store.NativeFSLockFactory@64288644) fullCopy=false
[junit4:junit4] 2> 4154812 T7392 C1952 P51422 oasc.SolrCore.execute [collection1] webapp= path=/replication params={file=segments_2&command=filecontent&checksum=true&wt=filestream&qt=/replication&generation=2} status=0 QTime=1
[junit4:junit4] 2> 4154817 T7439 C1951 P51429 oash.SnapPuller.fetchLatestIndex Total time taken for download : 0 secs
[junit4:junit4] 2> 4154819 T7439 C1951 P51429 oasu.DefaultSolrCoreState.newIndexWriter Creating new IndexWriter...
[junit4:junit4] 2> 4154820 T7439 C1951 P51429 oasu.DefaultSolrCoreState.newIndexWriter Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 2> 4154821 T7439 C1951 P51429 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@474e8210 lockFactory=org.apache.lucene.store.NativeFSLockFactory@604033a4),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@474e8210 lockFactory=org.apache.lucene.store.NativeFSLockFactory@604033a4),segFN=segments_2,generation=2}
[junit4:junit4] 2> 4154822 T7439 C1951 P51429 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 4154823 T7439 C1951 P51429 oasu.DefaultSolrCoreState.newIndexWriter New IndexWriter is ready to be used.
[junit4:junit4] 2> 4154823 T7439 C1951 P51429 oass.SolrIndexSearcher.<init> Opening Searcher@77fcfa57 main
[junit4:junit4] 2> 4154825 T7438 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@77fcfa57 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 4154827 T7439 C1951 P51429 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty4/index.20130529185825946 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty4/index.20130529185825946;done=true>>]
[junit4:junit4] 2> 4154827 T7439 C1951 P51429 oasc.CachingDirectoryFactory.close Closing directory: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty4/index.20130529185825946
[junit4:junit4] 2> 4154836 T7439 C1951 P51429 oasc.CachingDirectoryFactory.closeCacheValue Removing directory before core close: ./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1369879070344/jetty4/index.20130529185825946
[junit4:junit4] 2> 4154836 T7439 C1951 P51429 oasc.RecoveryStrategy.replay No replay needed. core=collection1
[junit4:junit4] 2> 4154837 T7439 C1951 P51429 oasc.RecoveryStrategy.doRecovery Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 2> 4154837 T7439 C1951 P51429 oasc.ZkController.publish publishing core=collection1 state=active
[junit4:junit4] 2> 4154838 T7439 C1951 P51429 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
[junit4:junit4] 2> 4154848 T7439 C1951 P51429 oasc.RecoveryStrategy.doRecovery Finished recovery process. core=collection1
[junit4:junit4] 2> 4155758 T7343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4156281 T7366 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4156283 T7366 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
[junit4:junit4] 2> "operation":"state",
[junit4:junit4] 2> "core_node_name":"127.0.0.1:51429__collection1",
[junit4:junit4] 2> "numShards":"2",
[junit4:junit4] 2> "shard":"shard2",
[junit4:junit4] 2> "roles":null,
[junit4:junit4] 2> "state":"active",
[junit4:junit4] 2> "shard_state":"active",
[junit4:junit4] 2> "core":"collection1",
[junit4:junit4] 2> "collection":"collection1",
[junit4:junit4] 2> "shard_range":null,
[junit4:junit4] 2> "node_name":"127.0.0.1:51429_",
[junit4:junit4] 2> "base_url":"http://127.0.0.1:51429"}
[junit4:junit4] 2> 4156295 T7385 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4156296 T7372 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4156296 T7365 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4156297 T7417 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4156297 T7401 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4156295 T7434 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 2> 4156764 T7343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4156767 T7343 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
[junit4:junit4] 2> ASYNC NEW_CORE C1953 name=collection1 org.apache.solr.core.SolrCore@4b77bfbb url=http://127.0.0.1:51415/collection1 node=127.0.0.1:51415_ C1953_STATE=coll:control_collection core:collection1 props:{shard=shard1, state=active, core=collection1, collection=control_collection, node_name=127.0.0.1:51415_, base_url=http://127.0.0.1:51415, leader=true}
[junit4:junit4] 2> 4156806 T7356 C1953 P51415 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 4156813 T7356 C1953 P51415 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@71bb44a3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@404f58f6),segFN=segments_1,generation=1}
[junit4:junit4] 2> 4156814 T7356 C1953 P51415 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
[junit4:junit4] 2> 4156816 T7356 C1953 P51415 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@71bb44a3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@404f58f6),segFN=segments_1,generation=1}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@71bb44a3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@404f58f6),segFN=segments_2,generation=2}
[junit4:junit4] 2> 4156817 T7356 C1953 P51415 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
[junit4:junit4] 2> 4156817 T7356 C1953 P51415 oass.SolrIndexSearcher.<init> Opening Searcher@54576132 main
[junit4:junit4] 2> 4156818 T7356 C1953 P51415 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 4156820 T7369 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@54576132 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 4156821 T7356 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 15
[junit4:junit4] 2> 4156854 T7392 C1952 P51422 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 4156856 T7392 C1952 P51422 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4d937b37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@47bb53a4),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4d937b37 lockFactory=org.apache.lucene.store.NativeFSLockFactory@47bb53a4),segFN=segments_3,generation=3}
[junit4:junit4] 2> 4156857 T7392 C1952 P51422 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 4156858 T7392 C1952 P51422 oass.SolrIndexSearcher.<init> Opening Searcher@6577906a main
[junit4:junit4] 2> 4156859 T7392 C1952 P51422 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 4156860 T7403 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6577906a main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 4156861 T7392 C1952 P51422 oasu.SolrCmdDistributor.distribCommit Distrib commit to:[StdNode: http://127.0.0.1:51419/collection1/, StdNode: http://127.0.0.1:51425/collection1/, StdNode: http://127.0.0.1:51429/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 2> 4156889 T7379 C1950 P51419 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C1949_STATE=coll:collection1 core:collection1 props:{shard=shard1, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51425_, base_url=http://127.0.0.1:51425}
[junit4:junit4] 2> 4156890 T7410 C1949 P51425 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> C1951_STATE=coll:collection1 core:collection1 props:{shard=shard2, state=active, core=collection1, collection=collection1, node_name=127.0.0.1:51429_, base_url=http://127.0.0.1:51429}
[junit4:junit4] 2> 4156897 T7428 C1951 P51429 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 2> 4156899 T7379 C1950 P51419 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@42d48dad lockFactory=org.apache.lucene.store.NativeFSLockFactory@4e86a408),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@42d48dad lockFactory=org.apache.lucene.store.NativeFSLockFactory@4e86a408),segFN=segments_3,generation=3}
[junit4:junit4] 2> 4156900 T7379 C1950 P51419 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 4156900 T7379 C1950 P51419 oass.SolrIndexSearcher.<init> Opening Searcher@12476a9c main
[junit4:junit4] 2> 4156901 T7379 C1950 P51419 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 4156906 T7387 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12476a9c main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 4156907 T7379 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 19
[junit4:junit4] 2> 4156908 T7410 C1949 P51425 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b77016b lockFactory=org.apache.lucene.store.NativeFSLockFactory@7511c1fd),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1b77016b lockFactory=org.apache.lucene.store.NativeFSLockFactory@7511c1fd),segFN=segments_3,generation=3}
[junit4:junit4] 2> 4156909 T7410 C1949 P51425 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 4156907 T7428 C1951 P51429 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@474e8210 lockFactory=org.apache.lucene.store.NativeFSLockFactory@604033a4),segFN=segments_2,generation=2}
[junit4:junit4] 2> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@474e8210 lockFactory=org.apache.lucene.store.NativeFSLockFactory@604033a4),segFN=segments_3,generation=3}
[junit4:junit4] 2> 4156912 T7428 C1951 P51429 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 3
[junit4:junit4] 2> 4156913 T7410 C1949 P51425 oass.SolrIndexSearcher.<init> Opening Searcher@6669b422 main
[junit4:junit4] 2> 4156914 T7410 C1949 P51425 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 4156915 T7428 C1951 P51429 oass.SolrIndexSearcher.<init> Opening Searcher@1e36e668 main
[junit4:junit4] 2> 4156915 T7419 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6669b422 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 4156917 T7438 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1e36e668 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 2> 4156918 T7428 C1951 P51429 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4:junit4] 2> 4156918 T7428 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 21
[junit4:junit4] 2> 4156919 T7410 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=false} {commit=} 0 29
[junit4:junit4] 2> 4156920 T7392 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={waitSearcher=true&commit=true&wt=javabin&version=2&softCommit=false} {commit=} 0 66
[junit4:junit4] 2> 4156923 T7343 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper...
[junit4:junit4] 2> 4156929 T7380 C1950 P51419 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 4156935 T7411 C1949 P51425 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 4156941 T7391 C1952 P51422 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=1
[junit4:junit4] 2> 4156958 T7429 C1951 P51429 oasc.SolrCore.execute [collection1] webapp= path=/select params={tests=checkShardConsistency&q=*:*&distrib=false&wt=javabin&version=2&rows=0} hits=0 status=0 QTime=0
[junit4:junit4] 2> 4158967 T7356 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1436422357758509056)} 0 4
[junit4:junit4] 2> 4158988 T7429 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1436422357768994816&update.from=http://127.0.0.1:51422/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1436422357768994816)} 0 7
[junit4:junit4] 2> 4159010 T7410 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=FROMLEADER&_version_=-1436422357777383424&update.from=http://127.0.0.1:51419/collection1/&wt=javabin&version=2} {deleteByQuery=*:* (-1436422357777383424)} 0 5
[junit4:junit4] 2> 4159012 T7380 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1436422357777383424)} 0 32
[junit4:junit4] 2> 4159014 T7392 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1436422357768994816)} 0 41
[junit4:junit4] 2> 4159022 T7357 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0 (1436422357816180736)]} 0 4
[junit4:junit4] 2> 4159053 T7411 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[0 (1436422357836103680)]} 0 11
[junit4:junit4] 2> 4159055 T7376 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[0 (1436422357836103680)]} 0 18
[junit4:junit4] 2> 4159064 T7393 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[0]} 0 30
[junit4:junit4] 2> 4159071 T7358 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1 (1436422357870706688)]} 0 1
[junit4:junit4] 2> 4159092 T7412 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[1 (1436422357885386752)]} 0 1
[junit4:junit4] 2> 4159096 T7381 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[1 (1436422357885386752)]} 0 11
[junit4:junit4] 2> 4159099 T7394 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[1]} 0 18
[junit4:junit4] 2> 4159109 T7359 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1436422357910552576)]} 0 1
[junit4:junit4] 2> 4159129 T7430 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[2 (1436422357916844032)]} 0 9
[junit4:junit4] 2> 4159132 T7395 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[2 (1436422357916844032)]} 0 18
[junit4:junit4] 2> 4159144 T7360 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1436422357948301312)]} 0 2
[junit4:junit4] 2> 4159157 T7425 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[3 (1436422357956689920)]} 0 1
[junit4:junit4] 2> 4159159 T7396 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[3 (1436422357956689920)]} 0 10
[junit4:junit4] 2> 4159164 T7361 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1436422357968224256)]} 0 1
[junit4:junit4] 2> 4159186 T7413 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[4 (1436422357983952896)]} 0 2
[junit4:junit4] 2> 4159188 T7378 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[4 (1436422357983952896)]} 0 10
[junit4:junit4] 2> 4159197 T7356 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1436422358002827264)]} 0 1
[junit4:junit4] 2> 4159223 T7426 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[5 (1436422358015410176)]} 0 1
[junit4:junit4] 2> 4159225 T7391 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[5 (1436422358015410176)]} 0 18
[junit4:junit4] 2> 4159230 T7357 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6 (1436422358037430272)]} 0 1
[junit4:junit4] 2> 4159248 T7427 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[6 (1436422358047916032)]} 0 6
[junit4:junit4] 2> 4159250 T7392 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[6 (1436422358047916032)]} 0 12
[junit4:junit4] 2> 4159251 T7377 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[6]} 0 17
[junit4:junit4] 2> 4159258 T7358 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7 (1436422358066790400)]} 0 2
[junit4:junit4] 2> 4159281 T7428 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[7 (1436422358084616192)]} 0 2
[junit4:junit4] 2> 4159283 T7393 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[7 (1436422358084616192)]} 0 9
[junit4:junit4] 2> 4159286 T7379 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[7]} 0 22
[junit4:junit4] 2> 4159295 T7360 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8 (1436422358106636288)]} 0 5
[junit4:junit4] 2> 4159308 T7408 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[8 (1436422358115024896)]} 0 1
[junit4:junit4] 2> 4159314 T7380 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[8 (1436422358115024896)]} 0 14
[junit4:junit4] 2> 4159318 T7394 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[8]} 0 20
[junit4:junit4] 2> 4159323 T7361 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1436422358135996416)]} 0 0
[junit4:junit4] 2> 4159335 T7429 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[9 (1436422358145433600)]} 0 0
[junit4:junit4] 2> 4159341 T7395 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[9 (1436422358145433600)]} 0 9
[junit4:junit4] 2> 4159352 T7356 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10 (1436422358165356544)]} 0 1
[junit4:junit4] 2> 4159380 T7409 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[10 (1436422358185279488)]} 0 1
[junit4:junit4] 2> 4159384 T7376 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[10 (1436422358185279488)]} 0 14
[junit4:junit4] 2> 4159391 T7396 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[10]} 0 31
[junit4:junit4] 2> 4159397 T7357 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1436422358212542464)]} 0 1
[junit4:junit4] 2> 4159408 T7411 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[11 (1436422358220931072)]} 0 1
[junit4:junit4] 2> 4159410 T7378 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[11 (1436422358220931072)]} 0 6
[junit4:junit4] 2> 4159422 T7358 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12 (1436422358239805440)]} 0 1
[junit4:junit4] 2> 4159434 T7412 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[12 (1436422358248194048)]} 0 1
[junit4:junit4] 2> 4159436 T7377 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[12 (1436422358248194048)]} 0 6
[junit4:junit4] 2> 4159439 T7392 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[12]} 0 12
[junit4:junit4] 2> 4159452 T7359 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1436422358271262720)]} 0 1
[junit4:junit4] 2> 4159463 T7413 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[13 (1436422358275457024)]} 0 4
[junit4:junit4] 2> 4159466 T7379 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[13 (1436422358275457024)]} 0 10
[junit4:junit4] 2> 4159471 T7360 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1436422358291185664)]} 0 0
[junit4:junit4] 2> 4159488 T7408 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[14 (1436422358296428544)]} 0 7
[junit4:junit4] 2> 4159491 T7380 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[14 (1436422358296428544)]} 0 15
[junit4:junit4] 2> 4159498 T7361 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15 (1436422358318448640)]} 0 1
[junit4:junit4] 2> 4159510 T7410 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[15 (1436422358327885824)]} 0 1
[junit4:junit4] 2> 4159511 T7381 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[15 (1436422358327885824)]} 0 6
[junit4:junit4] 2> 4159513 T7393 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[15]} 0 11
[junit4:junit4] 2> 4159518 T7356 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16 (1436422358339420160)]} 0 2
[junit4:junit4] 2> 4159528 T7409 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[16 (1436422358346760192)]} 0 1
[junit4:junit4] 2> 4159529 T7376 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[16 (1436422358346760192)]} 0 5
[junit4:junit4] 2> 4159531 T7394 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[16]} 0 10
[junit4:junit4] 2> 4159534 T7357 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1436422358357245952)]} 0 0
[junit4:junit4] 2> 4159541 T7430 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[17 (1436422358361440256)]} 0 1
[junit4:junit4] 2> 4159543 T7395 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[17 (1436422358361440256)]} 0 5
[junit4:junit4] 2> 4159547 T7358 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18 (1436422358369828864)]} 0 1
[junit4:junit4] 2> 4159556 T7426 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[18 (1436422358377168896)]} 0 0
[junit4:junit4] 2> 4159557 T7396 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[18 (1436422358377168896)]} 0 4
[junit4:junit4] 2> 4159559 T7378 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[18]} 0 9
[junit4:junit4] 2> 4159562 T7359 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19 (1436422358386606080)]} 0 0
[junit4:junit4] 2> 4159583 T7427 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[19 (1436422358399188992)]} 0 3
[junit4:junit4] 2> 4159585 T7391 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[19 (1436422358399188992)]} 0 11
[junit4:junit4] 2> 4159586 T7377 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[19]} 0 15
[junit4:junit4] 2> 4159594 T7360 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20 (1436422358419111936)]} 0 1
[junit4:junit4] 2> 4159601 T7411 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[20 (1436422358424354816)]} 0 0
[junit4:junit4] 2> 4159603 T7379 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[20 (1436422358424354816)]} 0 5
[junit4:junit4] 2> 4159607 T7361 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21 (1436422358432743424)]} 0 1
[junit4:junit4] 2> 4159621 T7428 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[21 (1436422358441132032)]} 0 0
[junit4:junit4] 2> 4159623 T7392 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[21 (1436422358441132032)]} 0 9
[junit4:junit4] 2> 4159626 T7380 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[21]} 0 15
[junit4:junit4] 2> 4159632 T7356 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22 (1436422358458957824)]} 0 1
[junit4:junit4] 2> 4159639 T7412 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[22 (1436422358463152128)]} 0 1
[junit4:junit4] 2> 4159640 T7381 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[22 (1436422358463152128)]} 0 5
[junit4:junit4] 2> 4159644 T7357 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23 (1436422358472589312)]} 0 1
[junit4:junit4] 2> 4159654 T7429 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[23 (1436422358478880768)]} 0 1
[junit4:junit4] 2> 4159663 T7393 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[23 (1436422358478880768)]} 0 13
[junit4:junit4] 2> 4159665 T7376 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[23]} 0 18
[junit4:junit4] 2> 4159678 T7358 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24 (1436422358507192320)]} 0 1
[junit4:junit4] 2> 4159698 T7413 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[24 (1436422358521872384)]} 0 1
[junit4:junit4] 2> 4159699 T7378 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[24 (1436422358521872384)]} 0 8
[junit4:junit4] 2> 4159701 T7394 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[24]} 0 17
[junit4:junit4] 2> 4159708 T7360 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25 (1436422358539698176)]} 0 0
[junit4:junit4] 2> 4159721 T7425 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[25 (1436422358547038208)]} 0 0
[junit4:junit4] 2> 4159723 T7395 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[25 (1436422358547038208)]} 0 8
[junit4:junit4] 2> 4159731 T7361 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26 (1436422358561718272)]} 0 3
[junit4:junit4] 2> 4159753 T7430 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[26 (1436422358581641216)]} 0 1
[junit4:junit4] 2> 4159754 T7396 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[26 (1436422358581641216)]} 0 6
[junit4:junit4] 2> 4159757 T7377 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[26]} 0 17
[junit4:junit4] 2> 4159764 T7356 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27 (1436422358598418432)]} 0 1
[junit4:junit4] 2> 4159782 T7408 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[27 (1436422358614147072)]} 0 0
[junit4:junit4] 2> 4159784 T7379 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[27 (1436422358614147072)]} 0 5
[junit4:junit4] 2> 4159788 T7391 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[27]} 0 12
[junit4:junit4] 2> 4159792 T7357 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28 (1436422358627778560)]} 0 1
[junit4:junit4] 2> 4159806 T7409 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[28 (1436422358631972864)]} 0 4
[junit4:junit4] 2> 4159809 T7381 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[28 (1436422358631972864)]} 0 13
[junit4:junit4] 2> 4159815 T7358 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1436422358649798656)]} 0 2
[junit4:junit4] 2> 4159825 T7426 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[29 (1436422358658187264)]} 0 1
[junit4:junit4] 2> 4159830 T7392 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[29 (1436422358658187264)]} 0 9
[junit4:junit4] 2> 4159840 T7359 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30 (1436422358677061632)]} 0 1
[junit4:junit4] 2> 4159851 T7427 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[30 (1436422358685450240)]} 0 1
[junit4:junit4] 2> 4159852 T7394 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[30 (1436422358685450240)]} 0 5
[junit4:junit4] 2> 4159855 T7376 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[30]} 0 11
[junit4:junit4] 2> 4159865 T7360 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31 (1436422358703276032)]} 0 1
[junit4:junit4] 2> 4159881 T7428 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[31 (1436422358712713216)]} 0 1
[junit4:junit4] 2> 4159883 T7395 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[31 (1436422358712713216)]} 0 12
[junit4:junit4] 2> 4159897 T7361 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32 (1436422358737879040)]} 0 1
[junit4:junit4] 2> 4159916 T7411 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[32 (1436422358748364800)]} 0 2
[junit4:junit4] 2> 4159918 T7378 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[32 (1436422358748364800)]} 0 11
[junit4:junit4] 2> 4159924 T7396 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[32]} 0 22
[junit4:junit4] 2> 4159935 T7356 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33 (1436422358776676352)]} 0 2
[junit4:junit4] 2> 4159949 T7429 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[33 (1436422358786113536)]} 0 1
[junit4:junit4] 2> 4159951 T7391 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[33 (1436422358786113536)]} 0 8
[junit4:junit4] 2> 4159960 T7357 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34 (1436422358803939328)]} 0 0
[junit4:junit4] 2> 4159974 T7412 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[34 (1436422358812327936)]} 0 1
[junit4:junit4] 2> 4159975 T7377 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[34 (1436422358812327936)]} 0 7
[junit4:junit4] 2> 4159989 T7358 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35 (1436422358832250880)]} 0 3
[junit4:junit4] 2> 4160008 T7413 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[35 (1436422358843785216)]} 0 4
[junit4:junit4] 2> 4160010 T7380 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[35 (1436422358843785216)]} 0 12
[junit4:junit4] 2> 4160016 T7359 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36 (1436422358861611008)]} 0 1
[junit4:junit4] 2> 4160035 T7408 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[36 (1436422358877339648)]} 0 1
[junit4:junit4] 2> 4160037 T7379 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[36 (1436422358877339648)]} 0 7
[junit4:junit4] 2> 4160046 T7393 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[36]} 0 25
[junit4:junit4] 2> 4160057 T7360 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37 (1436422358904602624)]} 0 1
[junit4:junit4] 2> 4160067 T7430 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[37 (1436422358910894080)]} 0 1
[junit4:junit4] 2> 4160070 T7392 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[37 (1436422358910894080)]} 0 9
[junit4:junit4] 2> 4160087 T7361 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38 (1436422358936059904)]} 0 5
[junit4:junit4] 2> 4160099 T7410 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[38 (1436422358946545664)]} 0 0
[junit4:junit4] 2> 4160101 T7381 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[38 (1436422358946545664)]} 0 6
[junit4:junit4] 2> 4160102 T7394 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[38]} 0 10
[junit4:junit4] 2> 4160107 T7357 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39 (1436422358957031424)]} 0 1
[junit4:junit4] 2> 4160116 T7426 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[39 (1436422358962274304)]} 0 0
[junit4:junit4] 2> 4160117 T7395 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[39 (1436422358962274304)]} 0 6
[junit4:junit4] 2> 4160126 T7358 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40 (1436422358976954368)]} 0 1
[junit4:junit4] 2> 4160147 T7409 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[40 (1436422358988488704)]} 0 5
[junit4:junit4] 2> 4160155 T7376 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[40 (1436422358988488704)]} 0 19
[junit4:junit4] 2> 4160160 T7396 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[40]} 0 29
[junit4:junit4] 2> 4160176 T7359 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41 (1436422359028334592)]} 0 3
[junit4:junit4] 2> 4160196 T7411 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[41 (1436422359039868928)]} 0 6
[junit4:junit4] 2> 4160197 T7378 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[41 (1436422359039868928)]} 0 13
[junit4:junit4] 2> 4160199 T7391 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[41]} 0 18
[junit4:junit4] 2> 4160208 T7360 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42 (1436422359058743296)]} 0 5
[junit4:junit4] 2> 4160217 T7413 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[42 (1436422359068180480)]} 0 1
[junit4:junit4] 2> 4160220 T7380 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[42 (1436422359068180480)]} 0 8
[junit4:junit4] 2> 4160231 T7356 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43 (1436422359088103424)]} 0 5
[junit4:junit4] 2> 4160257 T7408 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[43 (1436422359103832064)]} 0 1
[junit4:junit4] 2> 4160258 T7379 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[43 (1436422359103832064)]} 0 14
[junit4:junit4] 2> 4160259 T7393 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[43]} 0 24
[junit4:junit4] 2> 4160263 T7361 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44 (1436422359121657856)]} 0 0
[junit4:junit4] 2> 4160279 T7410 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[44 (1436422359135289344)]} 0 0
[junit4:junit4] 2> 4160281 T7381 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[44 (1436422359135289344)]} 0 5
[junit4:junit4] 2> 4160282 T7394 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[44]} 0 12
[junit4:junit4] 2> 4160294 T7357 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45 (1436422359153115136)]} 0 1
[junit4:junit4] 2> 4160314 T7427 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[45 (1436422359166746624)]} 0 1
[junit4:junit4] 2> 4160317 T7395 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[45 (1436422359166746624)]} 0 12
[junit4:junit4] 2> 4160324 T7358 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46 (1436422359184572416)]} 0 1
[junit4:junit4] 2> 4160336 T7409 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[46 (1436422359191912448)]} 0 1
[junit4:junit4] 2> 4160338 T7376 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[46 (1436422359191912448)]} 0 8
[junit4:junit4] 2> 4160343 T7359 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47 (1436422359204495360)]} 0 1
[junit4:junit4] 2> 4160360 T7412 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[47 (1436422359212883968)]} 0 5
[junit4:junit4] 2> 4160367 T7377 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[47 (1436422359212883968)]} 0 18
[junit4:junit4] 2> 4160372 T7360 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48 (1436422359235952640)]} 0 0
[junit4:junit4] 2> 4160393 T7411 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[48 (1436422359247486976)]} 0 1
[junit4:junit4] 2> 4160395 T7378 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[48 (1436422359247486976)]} 0 14
[junit4:junit4] 2> 4160405 T7356 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49 (1436422359269507072)]} 0 1
[junit4:junit4] 2> 4160443 T7413 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[49 (1436422359294672896)]} 0 1
[junit4:junit4] 2> 4160445 T7380 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[49 (1436422359294672896)]} 0 17
[junit4:junit4] 2> 4160447 T7396 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[49]} 0 33
[junit4:junit4] 2> 4160462 T7361 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50 (1436422359319838720)]} 0 11
[junit4:junit4] 2> 4160475 T7408 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[50 (1436422359339761664)]} 0 1
[junit4:junit4] 2> 4160476 T7379 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[50 (1436422359339761664)]} 0 6
[junit4:junit4] 2> 4160478 T7391 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[50]} 0 11
[junit4:junit4] 2> 4160482 T7357 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51 (1436422359351296000)]} 0 1
[junit4:junit4] 2> 4160493 T7410 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[51 (1436422359358636032)]} 0 1
[junit4:junit4] 2> 4160496 T7381 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[51 (1436422359358636032)]} 0 7
[junit4:junit4] 2> 4160504 T7392 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[51]} 0 18
[junit4:junit4] 2> 4160512 T7359 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52 (1436422359381704704)]} 0 1
[junit4:junit4] 2> 4160521 T7409 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[52 (1436422359386947584)]} 0 1
[junit4:junit4] 2> 4160523 T7376 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[52 (1436422359386947584)]} 0 7
[junit4:junit4] 2> 4160528 T7360 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1436422359398481920)]} 0 1
[junit4:junit4] 2> 4160535 T7428 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[53 (1436422359403724800)]} 0 0
[junit4:junit4] 2> 4160537 T7393 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[53 (1436422359403724800)]} 0 6
[junit4:junit4] 2> 4160541 T7356 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54 (1436422359413161984)]} 0 0
[junit4:junit4] 2> 4160556 T7412 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[54 (1436422359419453440)]} 0 2
[junit4:junit4] 2> 4160557 T7377 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[54 (1436422359419453440)]} 0 10
[junit4:junit4] 2> 4160558 T7394 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[54]} 0 14
[junit4:junit4] 2> 4160565 T7361 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55 (1436422359438327808)]} 0 1
[junit4:junit4] 2> 4160576 T7411 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[55 (1436422359445667840)]} 0 1
[junit4:junit4] 2> 4160577 T7378 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[55 (1436422359445667840)]} 0 5
[junit4:junit4] 2> 4160578 T7395 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[55]} 0 9
[junit4:junit4] 2> 4160584 T7358 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1436422359458250752)]} 0 0
[junit4:junit4] 2> 4160592 T7429 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[56 (1436422359462445056)]} 0 1
[junit4:junit4] 2> 4160594 T7396 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[56 (1436422359462445056)]} 0 6
[junit4:junit4] 2> 4160598 T7357 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57 (1436422359472930816)]} 0 1
[junit4:junit4] 2> 4160608 T7430 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[57 (1436422359479222272)]} 0 1
[junit4:junit4] 2> 4160609 T7391 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[57 (1436422359479222272)]} 0 5
[junit4:junit4] 2> 4160610 T7380 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[57]} 0 9
[junit4:junit4] 2> 4160614 T7359 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58 (1436422359488659456)]} 0 1
[junit4:junit4] 2> 4160627 T7426 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[58 (1436422359493902336)]} 0 0
[junit4:junit4] 2> 4160629 T7392 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[58 (1436422359493902336)]} 0 12
[junit4:junit4] 2> 4160633 T7360 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59 (1436422359509630976)]} 0 0
[junit4:junit4] 2> 4160641 T7413 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[59 (1436422359513825280)]} 0 1
[junit4:junit4] 2> 4160645 T7381 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[59 (1436422359513825280)]} 0 7
[junit4:junit4] 2> 4160650 T7356 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60 (1436422359527456768)]} 0 0
[junit4:junit4] 2> 4160660 T7427 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[60 (1436422359531651072)]} 0 1
[junit4:junit4] 2> 4160661 T7393 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[60 (1436422359531651072)]} 0 7
[junit4:junit4] 2> 4160666 T7361 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61 (1436422359543185408)]} 0 1
[junit4:junit4] 2> 4160674 T7410 C1949 P51425 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51419/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[61 (1436422359548428288)]} 0 1
[junit4:junit4] 2> 4160675 T7376 C1950 P51419 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[61 (1436422359548428288)]} 0 6
[junit4:junit4] 2> 4160679 T7358 C1953 P51415 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={wt=javabin&version=2} {add=[62 (1436422359557865472)]} 0 0
[junit4:junit4] 2> 4160697 T7425 C1951 P51429 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=http://127.0.0.1:51422/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[62 (1436422359572545536)]} 0 1
[junit4:junit4] 2> 4160698 T7395 C1952 P51422 oasup.LogUpdateProcessor.finish [collection1] webapp= path=/update params={distrib.from=h
[...truncated too long message...]
it4] 2> 4292248 T7506 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/dhzk,null}
[junit4:junit4] 2> 4292351 T7506 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 4292358 T7506 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51491 51491
[junit4:junit4] 2> 4292476 T7506 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 51494
[junit4:junit4] 2> !!!! WARNING: best effort to remove /Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ShardSplitTest-1369879182579 FAILED !!!!!
[junit4:junit4] 2> 4292478 T7506 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
[junit4:junit4] 2> 4292479 T7506 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:51491 51491
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=67B7C8B21887FED7 -Dtests.slow=true -Dtests.locale=lv -Dtests.timezone=Eire -Dtests.file.encoding=US-ASCII
[junit4:junit4] ERROR 61.1s | ShardSplitTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Server at http://127.0.0.1:51498/dhzk returned non ok status:500, message:Server Error
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([67B7C8B21887FED7:E65146AA6FD89EEB]:0)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
[junit4:junit4] > at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.splitShard(ShardSplitTest.java:208)
[junit4:junit4] > at org.apache.solr.cloud.ShardSplitTest.doTest(ShardSplitTest.java:133)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:815)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 2> 4292499 T7506 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4:junit4] 2> 61081 T7505 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
[junit4:junit4] 2> NOTE: test params are: codec=CheapBastard, sim=DefaultSimilarity, locale=lv, timezone=Eire
[junit4:junit4] 2> NOTE: Mac OS X 10.8.3 x86_64/Oracle Corporation 1.7.0_21 (64-bit)/cpus=2,threads=3,free=366889464,total=437256192
[junit4:junit4] 2> NOTE: All tests run in this JVM: [HighlighterTest, TestQuerySenderListener, StatsComponentTest, TestDistributedGrouping, TestExtendedDismaxParser, TestDynamicFieldResource, TestZkChroot, CircularListTest, TestRandomFaceting, TestRecovery, EchoParamsTest, TestSurroundQueryParser, TestAddFieldRealTimeGet, TestJmxIntegration, PolyFieldTest, TestSort, PathHierarchyTokenizerFactoryTest, IndexSchemaTest, SampleTest, TestLazyCores, ZkCLITest, SimpleFacetsTest, TestXIncludeConfig, TestCopyFieldCollectionResource, TestJmxMonitoredMap, PeerSyncTest, TestIndexSearcher, SuggesterWFSTTest, BadComponentTest, TestPluginEnable, SuggesterTSTTest, TestSuggestSpellingConverter, CacheHeaderTest, TestFuzzyAnalyzedSuggestions, TestAtomicUpdateErrorCases, TestRealTimeGet, TestStressReorder, IndexBasedSpellCheckerTest, OutputWriterTest, UpdateRequestProcessorFactoryTest, TestBadConfig, TimeZoneUtilsTest, DirectUpdateHandlerTest, QueryParsingTest, RegexBoostProcessorTest, TestFieldResource, TestIndexingPerformance, SpatialFilterTest, UnloadDistributedZkTest, TestSolrQueryParserResource, NumericFieldsTest, TestSolrQueryParser, TestSchemaResource, RecoveryZkTest, PreAnalyzedUpdateProcessorTest, DistanceFunctionTest, FieldMutatingUpdateProcessorTest, TestSchemaNameResource, AliasIntegrationTest, SpellPossibilityIteratorTest, SolrIndexConfigTest, TestPropInjectDefaults, TestCSVLoader, TestDefaultSimilarityFactory, ZkControllerTest, TestNumberUtils, HighlighterConfigTest, TestSolrQueryParserDefaultOperatorResource, SchemaVersionSpecificBehaviorTest, SuggesterTest, ShowFileRequestHandlerTest, RAMDirectoryFactoryTest, TestFastWriter, SpellCheckCollatorTest, QueryElevationComponentTest, TestFunctionQuery, TermVectorComponentDistributedTest, TestSolrCoreProperties, BasicZkTest, AnalysisAfterCoreReloadTest, BasicDistributedZkTest, BinaryUpdateRequestHandlerTest, TestDFRSimilarityFactory, SortByFunctionTest, TestShardHandlerFactory, MBeansHandlerTest, ReturnFieldsTest, TestDynamicFieldCollectionResource, OverseerCollectionProcessorTest, IndexReaderFactoryTest, FullSolrCloudDistribCmdsTest, ResourceLoaderTest, ChaosMonkeySafeLeaderTest, SolrInfoMBeanTest, TestSystemIdResolver, TestCloudManagedSchema, TestCloudManagedSchemaAddField, TestClassNameShortening, CoreAdminHandlerTest, FileUtilsTest, TestDistributedSearch, ChaosMonkeyNothingIsSafeTest, SolrCoreTest, BadCopyFieldTest, TermVectorComponentTest, TestFastOutputStream, TestQueryUtils, UniqFieldsUpdateProcessorFactoryTest, TestLFUCache, TestPHPSerializedResponseWriter, TestQuerySenderNoQuery, AutoCommitTest, TestReplicationHandler, JSONWriterTest, SolrCmdDistributorTest, SolrTestCaseJ4Test, LegacyHTMLStripCharFilterTest, TestIBSimilarityFactory, HardAutoCommitTest, TestSolrIndexConfig, SolrRequestParserTest, SoftAutoCommitTest, OpenExchangeRatesOrgProviderTest, PreAnalyzedFieldTest, TestLMJelinekMercerSimilarityFactory, TestStressLucene, DistributedSpellCheckComponentTest, TestSolrDeletionPolicy2, DefaultValueUpdateProcessorTest, TestLMDirichletSimilarityFactory, ClusterStateUpdateTest, SystemInfoHandlerTest, PingRequestHandlerTest, TestCoreDiscovery, TestOmitPositions, CopyFieldTest, IndexSchemaRuntimeFieldTest, TestQueryTypes, JsonLoaderTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, LeaderElectionIntegrationTest, ShardRoutingTest, LeaderElectionTest, ZkSolrClientTest, TestRandomDVFaceting, ShardRoutingCustomTest, SliceStateUpdateTest, TestFaceting, TestHashPartitioner, TestJoin, TestMultiCoreConfBootstrap, TestReload, TestStressVersions, DistributedTermsComponentTest, TestRangeQuery, TestCoreContainer, TestSolr4Spatial, SpellCheckComponentTest, TestGroupingSearch, BadIndexSchemaTest, ConvertedLegacyTest, TestFiltering, BasicFunctionalityTest, CurrencyFieldOpenExchangeTest, DistributedQueryElevationComponentTest, CurrencyFieldXmlFileTest, SolrIndexSplitterTest, SimplePostToolTest, SignatureUpdateProcessorFactoryTest, SuggesterFSTTest, TestFoldingMultitermQuery, DocValuesTest, TestTrie, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, TestUpdate, TestWordDelimiterFilterFactory, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DocValuesMultiTest, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, DebugComponentTest, LukeRequestHandlerTest, DisMaxRequestHandlerTest, StandardRequestHandlerTest, TestWriterPerf, DirectSolrSpellCheckerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, FileBasedSpellCheckerTest, TermsComponentTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, DocumentBuilderTest, TestValueSourceCache, MoreLikeThisHandlerTest, RequiredFieldsTest, FieldAnalysisRequestHandlerTest, TestArbitraryIndexDir, FastVectorHighlighterTest, LoggingHandlerTest, SolrPluginUtilsTest, TestCollationField, TestCSVResponseWriter, TestAnalyzedSuggestions, TestPartialUpdateDeduplication, CSVRequestHandlerTest, TestComponentsName, SearchHandlerTest, CoreContainerCoreInitFailuresTest, TestBinaryResponseWriter, SOLR749Test, TestPropInject, UpdateParamsTest, AlternateDirectoryTest, ResponseLogComponentTest, TestStressRecovery, TestMergePolicyConfig, MultiTermTest, TestDocSet, TestBinaryField, TestSearchPerf, MinimalSchemaTest, TestElisionMultitermQuery, TestConfig, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestPhraseSuggestions, TestCharFilters, TestCodecSupport, SynonymTokenizerTest, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestBM25SimilarityFactory, ScriptEngineTest, URLClassifyProcessorTest, PluginInfoTest, TestFastLRUCache, DateMathParserTest, TestSolrXMLSerializer, PrimUtilsTest, DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, ClusterStateTest, TestSolrJ, TestLRUCache, TestUtils, TestDocumentBuilder, ZkNodePropsTest, SliceStateTest, UUIDFieldTest, TestRTGBase, ChaosMonkeyShardSplitTest, ShardSplitTest]
[junit4:junit4] Completed in 61.64s, 1 test, 1 error <<< FAILURES!
[...truncated 52 lines...]
BUILD FAILED
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:380: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:360: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/build.xml:39: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/build.xml:181: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/solr/common-build.xml:437: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:1240: The following error occurred while executing this line:
/Users/jenkins/jenkins-slave/workspace/Lucene-Solr-trunk-MacOSX/lucene/common-build.xml:884: There were test failures: 295 suites, 1232 tests, 2 errors, 18 ignored (6 assumptions)
Total time: 123 minutes 51 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.7.0 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure