You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2013/04/14 18:46:58 UTC
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-ea-b84) - Build #
5166 - Still Failing!
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/5166/
Java: 32bit/jdk1.8.0-ea-b84 -client -XX:+UseSerialGC
1 tests failed.
REGRESSION: org.apache.solr.cloud.ChaosMonkeyShardSplitTest.testDistribSearch
Error Message:
shard2 is not consistent. Got 48 from http://127.0.0.1:57945/i_/o/collection1lastClient and got 47 from http://127.0.0.1:60439/i_/o/collection1
Stack Trace:
java.lang.AssertionError: shard2 is not consistent. Got 48 from http://127.0.0.1:57945/i_/o/collection1lastClient and got 47 from http://127.0.0.1:60439/i_/o/collection1
at __randomizedtesting.SeedInfo.seed([CCC8219427EC4A7A:4D2EAF8C50B32A46]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:963)
at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:188)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 8936 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.ChaosMonkeyShardSplitTest
[junit4:junit4] 1> INFO - 2013-04-14 16:29:35.955; org.apache.solr.BaseDistributedSearchTestCase; Setting hostContext system property: /i_/o
[junit4:junit4] 1> INFO - 2013-04-14 16:29:35.959; org.apache.solr.SolrTestCaseJ4; ###Starting testDistribSearch
[junit4:junit4] 1> INFO - 2013-04-14 16:29:35.959; org.apache.solr.cloud.ZkTestServer; STARTING ZK TEST SERVER
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:35.960; org.apache.solr.cloud.ZkTestServer$ZKServerMain; Starting server
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.060; org.apache.solr.cloud.ZkTestServer; start zk server on port:48131
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.061; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.064; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1d87b1a name:ZooKeeperConnection Watcher:127.0.0.1:48131 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.064; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.064; org.apache.solr.common.cloud.SolrZkClient; makePath: /solr
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.067; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.068; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@11f8511 name:ZooKeeperConnection Watcher:127.0.0.1:48131/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.069; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.069; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.071; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/shards
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.073; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.074; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/shards
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.076; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.076; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.078; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.079; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.082; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.082; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.084; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.084; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/protwords.txt
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.086; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.087; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.089; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.090; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.092; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.093; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.095; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.096; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.098; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.099; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.165; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.176; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:41482
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.176; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.177; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.177; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365956976102
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.178; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365956976102/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.178; org.apache.solr.core.CoreContainer; New CoreContainer 25879301
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.178; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365956976102/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.179; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365956976102/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.199; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.200; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.200; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.201; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.201; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.201; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.202; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.202; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.203; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.203; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.210; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.225; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:48131/solr
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.226; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.227; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.228; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@54af27 name:ZooKeeperConnection Watcher:127.0.0.1:48131 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.228; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.230; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.232; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.233; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@205980 name:ZooKeeperConnection Watcher:127.0.0.1:48131/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.234; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.235; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/queue
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.239; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/collection-queue-work
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.241; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.242; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41482_i_%2Fo
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.243; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:41482_i_%2Fo
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.245; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer_elect/election
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.249; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer_elect/leader
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.250; org.apache.solr.cloud.Overseer; Overseer (id=89519356376580099-127.0.0.1:41482_i_%2Fo-n_0000000000) starting
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.252; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/queue-work
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.254; org.apache.solr.cloud.OverseerCollectionProcessor; Process current queue of collection creations
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.255; org.apache.solr.common.cloud.SolrZkClient; makePath: /clusterstate.json
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.256; org.apache.solr.common.cloud.SolrZkClient; makePath: /aliases.json
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.257; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.259; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Starting to work on the main queue
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.261; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365956976102/collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.262; org.apache.solr.cloud.ZkController; Check for collection zkNode:control_collection
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.262; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.262; org.apache.solr.cloud.ZkController; Load collection config from:/collections/control_collection
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.263; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365956976102/collection1/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.264; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365956976102/collection1/lib/README' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.264; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365956976102/collection1/lib/classes/' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.296; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.339; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.341; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.345; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.808; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.818; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.821; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.832; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.835; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.838; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 16:29:36.839; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 16:29:36.839; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.839; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 16:29:36.840; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 16:29:36.840; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.840; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.847; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-controljetty-1365956976102/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/control/data/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.847; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@126776
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.847; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.848; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/control/data
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.848; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/control/data/index/
[junit4:junit4] 1> WARN - 2013-04-14 16:29:36.848; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.849; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/control/data/index
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.852; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@9c34aa; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.852; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.854; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.854; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.854; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.855; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.855; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.856; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.856; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.856; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.857; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.859; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.862; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@a55720 main
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.863; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.863; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.867; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@a55720 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.868; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 16:29:36.868; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.761; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.762; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=1 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"1",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "shard":null,
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:41482_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "core_node_name":"1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:41482/i_/o"}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.762; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Create collection control_collection with numShards 1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.763; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.765; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.870; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.871; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:41482/i_/o collection:control_collection shard:shard1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.872; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.876; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.878; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.878; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.878; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:41482/i_/o/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.878; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.879; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:41482/i_/o/collection1/ has no replicas
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.879; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:41482/i_/o/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:37.879; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.268; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.274; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.286; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:41482/i_/o/collection1/ and leader is http://127.0.0.1:41482/i_/o/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.286; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:41482/i_/o
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.287; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.287; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.287; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.288; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.289; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.290; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.290; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.295; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.296; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.302; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@b27d05 name:ZooKeeperConnection Watcher:127.0.0.1:48131/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.302; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.303; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.305; org.apache.solr.cloud.ChaosMonkey; monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.380; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.382; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:57945
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.383; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.383; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.383; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365956979306
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.384; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365956979306/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.384; org.apache.solr.core.CoreContainer; New CoreContainer 1374826
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.384; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365956979306/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.385; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365956979306/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.405; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.406; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.406; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.406; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.407; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.407; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.407; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.408; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.408; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.409; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.421; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.432; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:48131/solr
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.433; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.434; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.435; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@facee9 name:ZooKeeperConnection Watcher:127.0.0.1:48131 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.435; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.438; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.441; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.442; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1780b2c name:ZooKeeperConnection Watcher:127.0.0.1:48131/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.442; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:39.445; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.447; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57945_i_%2Fo
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.449; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:57945_i_%2Fo
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.451; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.451; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.451; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.452; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.456; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365956979306/collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.456; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.457; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.457; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.458; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365956979306/collection1/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.458; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365956979306/collection1/lib/README' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.459; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365956979306/collection1/lib/classes/' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.493; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.532; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.534; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.538; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.779; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.780; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=1 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"1",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:41482_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "core_node_name":"1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:41482/i_/o"}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.784; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.784; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:40.784; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.015; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.025; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.028; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.040; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.043; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.046; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 16:29:41.047; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 16:29:41.047; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.047; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 16:29:41.048; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 16:29:41.048; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.048; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.049; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty1-1365956979306/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty1/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.049; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@126776
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.050; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.050; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.051; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty1/index/
[junit4:junit4] 1> WARN - 2013-04-14 16:29:41.051; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.051; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty1/index
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.208; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@bf165c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.209; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.210; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.211; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.211; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.212; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.213; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.213; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.213; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.214; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.214; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.217; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.220; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@ed9e75 main
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.220; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.221; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.229; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@ed9e75 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.231; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 16:29:41.232; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 16:29:42.287; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:42.288; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "shard":null,
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:57945_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"2",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:57945/i_/o"}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:42.288; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Create collection collection1 with numShards 2
[junit4:junit4] 1> INFO - 2013-04-14 16:29:42.288; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard2
[junit4:junit4] 1> INFO - 2013-04-14 16:29:42.350; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:42.350; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:42.350; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.233; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.233; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:57945/i_/o collection:collection1 shard:shard2
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.234; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.242; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.244; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.244; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.245; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:57945/i_/o/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.245; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.245; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:57945/i_/o/collection1/ has no replicas
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.245; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:57945/i_/o/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.246; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.853; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.927; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.927; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.927; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.951; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:57945/i_/o/collection1/ and leader is http://127.0.0.1:57945/i_/o/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.952; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57945/i_/o
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.952; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.953; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.953; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.956; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.957; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.958; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 16:29:43.958; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.027; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.029; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:40562
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.030; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.030; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.030; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365956983961
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.031; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365956983961/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.031; org.apache.solr.core.CoreContainer; New CoreContainer 15537862
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.032; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365956983961/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.032; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365956983961/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.060; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.061; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.061; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.062; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.062; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.062; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.063; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.063; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.063; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.064; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.071; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.088; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:48131/solr
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.089; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.090; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.091; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@14733d0 name:ZooKeeperConnection Watcher:127.0.0.1:48131 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.091; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.093; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.096; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.097; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@6f20c6 name:ZooKeeperConnection Watcher:127.0.0.1:48131/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.097; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:44.100; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.102; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40562_i_%2Fo
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.103; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:40562_i_%2Fo
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.105; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.105; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.106; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.106; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.106; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.106; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.107; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.113; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365956983961/collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.113; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.114; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.114; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.116; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365956983961/collection1/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.117; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365956983961/collection1/lib/README' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.117; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365956983961/collection1/lib/classes/' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.162; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.216; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.217; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.223; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.433; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.434; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:57945_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"2",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:57945/i_/o"}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.436; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.437; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.437; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.436; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.694; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.700; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.703; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.719; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.722; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.725; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 16:29:45.726; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 16:29:45.726; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.727; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 16:29:45.728; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 16:29:45.728; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.728; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.728; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty2-1365956983961/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty2/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.729; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@126776
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.729; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.729; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty2
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.730; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty2/index/
[junit4:junit4] 1> WARN - 2013-04-14 16:29:45.730; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.730; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty2/index
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.733; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a373e6; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.734; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.742; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.743; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.744; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.745; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.746; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.746; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.747; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.748; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.748; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.753; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.756; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@1c4e894 main
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.756; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.757; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.763; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@1c4e894 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.765; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 16:29:45.765; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 16:29:46.940; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:46.941; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "shard":null,
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:40562_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"3",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:40562/i_/o"}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:46.941; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Collection already exists with numShards=2
[junit4:junit4] 1> INFO - 2013-04-14 16:29:46.941; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:46.944; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:46.945; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:46.944; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:46.945; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:47.767; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:47.767; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:40562/i_/o collection:collection1 shard:shard1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:47.768; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 1> INFO - 2013-04-14 16:29:47.772; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:47.775; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:47.775; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4] 1> INFO - 2013-04-14 16:29:47.775; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:40562/i_/o/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:47.775; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4] 1> INFO - 2013-04-14 16:29:47.776; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:40562/i_/o/collection1/ has no replicas
[junit4:junit4] 1> INFO - 2013-04-14 16:29:47.776; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:40562/i_/o/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:47.776; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.448; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.456; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.457; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.456; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.456; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.481; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:40562/i_/o/collection1/ and leader is http://127.0.0.1:40562/i_/o/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.482; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40562/i_/o
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.482; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.482; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.483; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.485; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.486; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.487; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.487; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.576; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.579; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:60439
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.579; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.580; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.580; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365956988494
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.581; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365956988494/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.582; org.apache.solr.core.CoreContainer; New CoreContainer 20893738
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.582; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365956988494/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.582; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365956988494/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.604; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.604; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.605; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.605; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.605; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.606; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.606; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.606; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.607; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.607; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.614; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.634; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:48131/solr
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.635; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.636; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.638; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1ce78d name:ZooKeeperConnection Watcher:127.0.0.1:48131 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.638; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.640; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.643; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.644; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1b09d5f name:ZooKeeperConnection Watcher:127.0.0.1:48131/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.644; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:48.647; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.649; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60439_i_%2Fo
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.651; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:60439_i_%2Fo
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.652; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.653; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.653; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.653; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.654; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.654; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.654; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.654; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.654; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.659; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365956988494/collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.660; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.660; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.660; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.661; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365956988494/collection1/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.662; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365956988494/collection1/lib/README' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.662; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365956988494/collection1/lib/classes/' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.699; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.824; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.825; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.830; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.961; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.962; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:40562_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"3",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:40562/i_/o"}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.965; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.965; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.965; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.965; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:49.966; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.287; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.293; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.296; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.313; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.316; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.320; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 16:29:50.321; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 16:29:50.321; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.322; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 16:29:50.323; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 16:29:50.323; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.324; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.324; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty3-1365956988494/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.324; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@126776
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.325; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.325; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.325; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index/
[junit4:junit4] 1> WARN - 2013-04-14 16:29:50.326; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.326; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.330; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@c2eb2f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.330; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.332; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.332; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.333; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.333; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.334; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.334; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.334; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.335; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.335; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.338; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.341; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@13e3abe main
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.342; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.342; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.348; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@13e3abe main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.350; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 16:29:50.351; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 16:29:51.469; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:51.470; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "shard":null,
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:60439_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"4",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:60439/i_/o"}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:51.470; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Collection already exists with numShards=2
[junit4:junit4] 1> INFO - 2013-04-14 16:29:51.470; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard2
[junit4:junit4] 1> INFO - 2013-04-14 16:29:51.473; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:51.473; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:51.473; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:51.473; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:51.473; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.352; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.353; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:60439/i_/o collection:collection1 shard:shard2
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.357; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:60439/i_/o/collection1/ and leader is http://127.0.0.1:57945/i_/o/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.357; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:60439/i_/o
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.357; org.apache.solr.cloud.ZkController; Core needs to recover:collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.357; org.apache.solr.update.DefaultSolrCoreState; Running recovery - first canceling any ongoing recovery
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.358; org.apache.solr.cloud.RecoveryStrategy; Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.358; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.358; org.apache.solr.cloud.RecoveryStrategy; ###### startupVersions=[]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.359; org.apache.solr.cloud.ZkController; publishing core=collection1 state=recovering
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.359; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.360; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.360; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.360; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.361; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.366; org.apache.solr.handler.admin.CoreAdminHandler; Going to wait for coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.427; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.434; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:35433
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.435; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.436; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.436; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365956992364
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.436; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365956992364/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.437; org.apache.solr.core.CoreContainer; New CoreContainer 12181547
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.437; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365956992364/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.438; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365956992364/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.458; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.459; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.459; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.459; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.460; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.460; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.461; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.461; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.461; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.462; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.471; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.492; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:48131/solr
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.492; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.493; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.494; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1e0043d name:ZooKeeperConnection Watcher:127.0.0.1:48131 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.495; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.496; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.499; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.500; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@2513ce name:ZooKeeperConnection Watcher:127.0.0.1:48131/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.501; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.503; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.977; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.978; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "state":"recovering",
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:60439_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"4",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:60439/i_/o"}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.982; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.983; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.983; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.982; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.982; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:52.982; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.367; org.apache.solr.handler.admin.CoreAdminHandler; Waited coreNodeName: 4, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.367; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={coreNodeName=4&state=recovering&checkLive=true&nodeName=127.0.0.1:60439_i_%252Fo&action=PREPRECOVERY&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.505; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35433_i_%2Fo
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.507; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:35433_i_%2Fo
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.509; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.509; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.510; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.509; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.510; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.510; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.510; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.509; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.511; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.511; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.511; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.512; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.516; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365956992364/collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.516; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.517; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.517; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.518; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365956992364/collection1/'
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.519; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365956992364/collection1/lib/README' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.519; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365956992364/collection1/lib/classes/' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.559; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.602; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.603; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:53.609; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.083; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.098; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.102; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.123; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.130; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.135; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 16:29:54.137; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 16:29:54.137; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.138; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 16:29:54.139; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 16:29:54.139; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.140; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.140; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-jetty4-1365956992364/collection1/, dataDir=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.140; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@126776
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.141; org.apache.solr.core.SolrCore; solr.NRTCachingDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.142; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.142; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index/
[junit4:junit4] 1> WARN - 2013-04-14 16:29:54.143; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.143; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.147; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18e38e7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.148; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.150; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.150; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.151; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.151; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.152; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.152; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.153; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.154; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.155; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.158; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.162; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@1ecc549 main
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.163; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.163; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.169; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@1ecc549 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.171; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.171; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.488; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.489; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "shard":null,
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:35433_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"5",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:35433/i_/o"}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.490; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Collection already exists with numShards=2
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.490; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.495; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.496; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.495; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.496; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.496; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:54.496; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.173; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.173; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:35433/i_/o collection:collection1 shard:shard1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.176; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:35433/i_/o/collection1/ and leader is http://127.0.0.1:40562/i_/o/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.177; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35433/i_/o
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.177; org.apache.solr.cloud.ZkController; Core needs to recover:collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.177; org.apache.solr.update.DefaultSolrCoreState; Running recovery - first canceling any ongoing recovery
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.178; org.apache.solr.cloud.RecoveryStrategy; Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.179; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.179; org.apache.solr.cloud.RecoveryStrategy; ###### startupVersions=[]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.179; org.apache.solr.cloud.ZkController; publishing core=collection1 state=recovering
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.180; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.180; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.181; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.181; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.182; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.186; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.186; org.apache.solr.handler.admin.CoreAdminHandler; Going to wait for coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.187; org.apache.solr.cloud.AbstractFullDistribZkTestBase; Wait for recoveries to finish - wait 15 for each attempt
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.187; org.apache.solr.cloud.AbstractDistribZkTestBase; Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):15
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.188; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.368; org.apache.solr.cloud.RecoveryStrategy; Attempting to PeerSync from http://127.0.0.1:57945/i_/o/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.369; org.apache.solr.update.PeerSync; PeerSync: core=collection1 url=http://127.0.0.1:60439/i_/o START replicas=[http://127.0.0.1:57945/i_/o/collection1/] nUpdates=100
[junit4:junit4] 1> WARN - 2013-04-14 16:29:55.370; org.apache.solr.update.PeerSync; no frame of reference to tell of we've missed updates
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.370; org.apache.solr.cloud.RecoveryStrategy; PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.371; org.apache.solr.cloud.RecoveryStrategy; Starting Replication Recovery. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.371; org.apache.solr.cloud.RecoveryStrategy; Begin buffering updates. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.372; org.apache.solr.update.UpdateLog; Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.372; org.apache.solr.cloud.RecoveryStrategy; Attempting to replicate from http://127.0.0.1:57945/i_/o/collection1/. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.372; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.389; org.apache.solr.core.SolrCore; [collection1] webapp=/i_/o path=/get params={getVersions=100&qt=/get&wt=javabin&version=2&distrib=false} status=0 QTime=1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.389; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.393; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@bf165c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.393; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.407; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@bf165c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@bf165c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.408; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.409; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@198a795 realtime
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.410; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.411; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={commit=true&openSearcher=false&waitSearcher=true&softCommit=false&wt=javabin&version=2&commit_end_point=true} {commit=} 0 22
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.412; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for 10000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.413; org.apache.solr.handler.SnapPuller; No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.420; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for 10000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.421; org.apache.solr.core.SolrCore; [collection1] webapp=/i_/o path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=6
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.422; org.apache.solr.handler.SnapPuller; Master's generation: 2
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.422; org.apache.solr.handler.SnapPuller; Slave's generation: 1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.422; org.apache.solr.handler.SnapPuller; Starting replication process
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.425; org.apache.solr.core.SolrCore; [collection1] webapp=/i_/o path=/replication params={generation=2&command=filelist&qt=/replication&wt=javabin&version=2} status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.426; org.apache.solr.handler.SnapPuller; Number of files in latest index in master: 1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.426; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index.20130414122955426
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.427; org.apache.solr.handler.SnapPuller; Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index.20130414122955426 lockFactory=org.apache.lucene.store.NativeFSLockFactory@126e0b1; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.429; org.apache.solr.core.SolrCore; [collection1] webapp=/i_/o path=/replication params={file=segments_2&generation=2&command=filecontent&qt=/replication&checksum=true&wt=filestream} status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.431; org.apache.solr.handler.SnapPuller; Total time taken for download : 0 secs
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.438; org.apache.solr.update.DefaultSolrCoreState; Creating new IndexWriter...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.438; org.apache.solr.update.DefaultSolrCoreState; Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.440; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@c2eb2f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@c2eb2f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.441; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.441; org.apache.solr.update.DefaultSolrCoreState; New IndexWriter is ready to be used.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.442; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@f60929 main
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.443; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@f60929 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.444; org.apache.solr.core.CachingDirectoryFactory; looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index.20130414122955426 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index.20130414122955426;done=true>>]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.444; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index.20130414122955426
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.444; org.apache.solr.core.CachingDirectoryFactory; Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index.20130414122955426
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.447; org.apache.solr.cloud.RecoveryStrategy; No replay needed. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.447; org.apache.solr.cloud.RecoveryStrategy; Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.448; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.448; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 16:29:55.450; org.apache.solr.cloud.RecoveryStrategy; Finished recovery process. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:56.002; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:56.003; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "state":"recovering",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:35433_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"5",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:35433/i_/o"}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:56.008; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:60439_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"4",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:60439/i_/o"}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:56.027; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:56.027; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:56.027; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:56.027; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:56.027; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:56.027; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:56.187; org.apache.solr.handler.admin.CoreAdminHandler; Waited coreNodeName: 5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:56.187; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={coreNodeName=5&state=recovering&checkLive=true&nodeName=127.0.0.1:35433_i_%252Fo&action=PREPRECOVERY&core=collection1&wt=javabin&onlyIfLeader=true&version=2} status=0 QTime=1001
[junit4:junit4] 1> INFO - 2013-04-14 16:29:56.189; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:57.190; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.188; org.apache.solr.cloud.RecoveryStrategy; Attempting to PeerSync from http://127.0.0.1:40562/i_/o/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.189; org.apache.solr.update.PeerSync; PeerSync: core=collection1 url=http://127.0.0.1:35433/i_/o START replicas=[http://127.0.0.1:40562/i_/o/collection1/] nUpdates=100
[junit4:junit4] 1> WARN - 2013-04-14 16:29:58.189; org.apache.solr.update.PeerSync; no frame of reference to tell of we've missed updates
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.190; org.apache.solr.cloud.RecoveryStrategy; PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.190; org.apache.solr.cloud.RecoveryStrategy; Starting Replication Recovery. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.191; org.apache.solr.cloud.RecoveryStrategy; Begin buffering updates. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.191; org.apache.solr.update.UpdateLog; Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.192; org.apache.solr.cloud.RecoveryStrategy; Attempting to replicate from http://127.0.0.1:40562/i_/o/collection1/. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.192; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.192; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.200; org.apache.solr.core.SolrCore; [collection1] webapp=/i_/o path=/get params={getVersions=100&qt=/get&wt=javabin&version=2&distrib=false} status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.203; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.207; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a373e6; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.208; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.217; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a373e6; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a373e6; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.218; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.219; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@10c744e realtime
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.220; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.220; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={commit=true&openSearcher=false&waitSearcher=true&softCommit=false&wt=javabin&version=2&commit_end_point=true} {commit=} 0 18
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.222; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for 10000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.223; org.apache.solr.handler.SnapPuller; No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.225; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for 10000
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.226; org.apache.solr.core.SolrCore; [collection1] webapp=/i_/o path=/replication params={command=indexversion&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.227; org.apache.solr.handler.SnapPuller; Master's generation: 2
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.227; org.apache.solr.handler.SnapPuller; Slave's generation: 1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.228; org.apache.solr.handler.SnapPuller; Starting replication process
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.231; org.apache.solr.core.SolrCore; [collection1] webapp=/i_/o path=/replication params={generation=2&command=filelist&qt=/replication&wt=javabin&version=2} status=0 QTime=1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.232; org.apache.solr.handler.SnapPuller; Number of files in latest index in master: 1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.233; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index.20130414122958232
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.234; org.apache.solr.handler.SnapPuller; Starting download to NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index.20130414122958232 lockFactory=org.apache.lucene.store.NativeFSLockFactory@17c515a; maxCacheMB=48.0 maxMergeSizeMB=4.0) fullCopy=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.238; org.apache.solr.core.SolrCore; [collection1] webapp=/i_/o path=/replication params={file=segments_2&generation=2&command=filecontent&qt=/replication&checksum=true&wt=filestream} status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.240; org.apache.solr.handler.SnapPuller; Total time taken for download : 0 secs
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.248; org.apache.solr.update.DefaultSolrCoreState; Creating new IndexWriter...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.248; org.apache.solr.update.DefaultSolrCoreState; Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.251; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18e38e7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18e38e7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.252; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.253; org.apache.solr.update.DefaultSolrCoreState; New IndexWriter is ready to be used.
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.253; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@8464b7 main
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.255; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@8464b7 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.256; org.apache.solr.core.CachingDirectoryFactory; looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index.20130414122958232 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index.20130414122958232;done=true>>]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.256; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index.20130414122958232
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.257; org.apache.solr.core.CachingDirectoryFactory; Removing directory before core close: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index.20130414122958232
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.257; org.apache.solr.cloud.RecoveryStrategy; No replay needed. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.257; org.apache.solr.cloud.RecoveryStrategy; Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.258; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.258; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 16:29:58.259; org.apache.solr.cloud.RecoveryStrategy; Finished recovery process. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.037; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.039; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:35433_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"5",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:35433/i_/o"}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.044; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.044; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.044; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.044; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.044; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.044; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.195; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.196; org.apache.solr.cloud.AbstractDistribZkTestBase; Recoveries finished - collection: collection1
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.215; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.218; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@9c34aa; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.218; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.247; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@9c34aa; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/control/data/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@9c34aa; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.248; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.249; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@11a2f6 main
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.250; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.252; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@11a2f6 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.253; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={commit=true&waitSearcher=true&softCommit=false&wt=javabin&version=2} {commit=} 0 38
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.261; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.277; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a373e6; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty2/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@1a373e6; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.278; org.apache.solr.core.SolrDeletionPolicy; newest commit = 3[segments_3]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.279; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@990400 main
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.280; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.281; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@990400 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.283; org.apache.solr.update.SolrCmdDistributor; Distrib commit to:[StdNode: http://127.0.0.1:35433/i_/o/collection1/, StdNode: http://127.0.0.1:57945/i_/o/collection1/, StdNode: http://127.0.0.1:60439/i_/o/collection1/] params:commit_end_point=true&commit=true&softCommit=false&waitSearcher=true&expungeDeletes=false
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.288; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.297; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@bf165c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty1/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@bf165c; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.298; org.apache.solr.core.SolrDeletionPolicy; newest commit = 3[segments_3]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.299; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@11ad122 main
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.300; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.302; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@11ad122 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.303; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={commit=true&waitSearcher=true&softCommit=false&wt=javabin&version=2&expungeDeletes=false&commit_end_point=true} {commit=} 0 15
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.303; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.303; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.327; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@c2eb2f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty3/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@c2eb2f; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.328; org.apache.solr.core.SolrDeletionPolicy; newest commit = 3[segments_3]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.328; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18e38e7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> commit{dir=NRTCachingDirectory(org.apache.lucene.store.NIOFSDirectory@/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index lockFactory=org.apache.lucene.store.NativeFSLockFactory@18e38e7; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_3,generation=3,filenames=[segments_3]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.329; org.apache.solr.core.SolrDeletionPolicy; newest commit = 3[segments_3]
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.329; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@1865aa main
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.330; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@7071cf main
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.330; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.331; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.331; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@1865aa main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.332; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={commit=true&waitSearcher=true&softCommit=false&wt=javabin&version=2&expungeDeletes=false&commit_end_point=true} {commit=} 0 29
[junit4:junit4] 1> WARN - 2013-04-14 16:29:59.332; org.apache.solr.core.JmxMonitoredMap; Failed to register info bean: searcher
[junit4:junit4] 1> javax.management.InstanceAlreadyExistsException: solr/collection1:id=org.apache.solr.search.SolrIndexSearcher,type=searcher
[junit4:junit4] 1> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
[junit4:junit4] 1> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
[junit4:junit4] 1> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
[junit4:junit4] 1> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
[junit4:junit4] 1> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
[junit4:junit4] 1> at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:513)
[junit4:junit4] 1> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:140)
[junit4:junit4] 1> at org.apache.solr.core.JmxMonitoredMap.put(JmxMonitoredMap.java:51)
[junit4:junit4] 1> at org.apache.solr.search.SolrIndexSearcher.register(SolrIndexSearcher.java:298)
[junit4:junit4] 1> at org.apache.solr.core.SolrCore.registerSearcher(SolrCore.java:1768)
[junit4:junit4] 1> at org.apache.solr.core.SolrCore.access$000(SolrCore.java:131)
[junit4:junit4] 1> at org.apache.solr.core.SolrCore$7.call(SolrCore.java:1645)
[junit4:junit4] 1> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[junit4:junit4] 1> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[junit4:junit4] 1> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[junit4:junit4] 1> at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.336; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@7071cf main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.336; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={commit=true&waitSearcher=true&softCommit=false&wt=javabin&version=2&expungeDeletes=false&commit_end_point=true} {commit=} 0 33
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.337; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={commit=true&waitSearcher=true&softCommit=false&wt=javabin&version=2} {commit=} 0 76
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.338; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.340; org.apache.solr.core.SolrCore; [collection1] webapp=/i_/o path=/select params={rows=0&tests=checkShardConsistency&q=*:*&wt=javabin&version=2&distrib=false} hits=0 status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.342; org.apache.solr.core.SolrCore; [collection1] webapp=/i_/o path=/select params={rows=0&tests=checkShardConsistency&q=*:*&wt=javabin&version=2&distrib=false} hits=0 status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.344; org.apache.solr.core.SolrCore; [collection1] webapp=/i_/o path=/select params={rows=0&tests=checkShardConsistency&q=*:*&wt=javabin&version=2&distrib=false} hits=0 status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 16:29:59.346; org.apache.solr.core.SolrCore; [collection1] webapp=/i_/o path=/select params={rows=0&tests=checkShardConsistency&q=*:*&wt=javabin&version=2&distrib=false} hits=0 status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.347; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.349; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1f2398b name:ZooKeeperConnection Watcher:127.0.0.1:48131 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.349; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> / (2)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr (7)
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/election (5)
[junit4:junit4] 1> /solr/overseer_elect/election/89519356376580108-127.0.0.1:35433_i_%2Fo-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89519356376580102-127.0.0.1:57945_i_%2Fo-n_0000000001 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89519356376580106-127.0.0.1:60439_i_%2Fo-n_0000000003 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89519356376580104-127.0.0.1:40562_i_%2Fo-n_0000000002 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89519356376580099-127.0.0.1:41482_i_%2Fo-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89519356376580099-127.0.0.1:41482_i_%2Fo-n_0000000000"}
[junit4:junit4] 1> /solr/configs (1)
[junit4:junit4] 1> /solr/configs/conf1 (9)
[junit4:junit4] 1> /solr/configs/conf1/solrconfig.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/schema.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/mapping-ISOLatin1Accent.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/synonyms.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/stopwords.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/protwords.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/currency.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/old_synonyms.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/open-exchange-rates.json (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "disclaimer": "This data is not real, it was synthetically created to match currency.xml. It is modeled after the data format available from openexchangerates.org. See https://openexchangerates.org/documentation for details",
[junit4:junit4] 1> "license": "http://www.apache.org/licenses/LICENSE-2.0",
[junit4:junit4] 1> "timestamp": 1332070464,
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> "IMPORTANT NOTE": "In order for tests to work, this data must be kept in sync with ./currency.xml",
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> "base": "USD",
[junit4:junit4] 1> "rates": {
[junit4:junit4] 1> "USD": 1,
[junit4:junit4] 1> "JPY": 81.29,
[junit4:junit4] 1> "EUR": 2.5,
[junit4:junit4] 1> "GBP": 0.5,
[junit4:junit4] 1> "MXN": 2.0
[junit4:junit4] 1> }
[junit4:junit4] 1> }
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/collections (2)
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (2)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "node_name":"127.0.0.1:40562_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:40562/i_/o"}
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard2 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "node_name":"127.0.0.1:57945_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:57945/i_/o"}
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (2)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (2)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89519356376580104-3-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89519356376580108-5-n_0000000001 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard2 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard2/election (2)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/89519356376580106-4-n_0000000001 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/89519356376580102-2-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "node_name":"127.0.0.1:41482_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:41482/i_/o"}
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89519356376580099-1-n_0000000000 (0)
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1> /solr/clusterstate.json (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "collection1":{
[junit4:junit4] 1> "shards":{
[junit4:junit4] 1> "shard1":{
[junit4:junit4] 1> "range":"80000000-ffffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{
[junit4:junit4] 1> "3":{
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "node_name":"127.0.0.1:40562_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:40562/i_/o",
[junit4:junit4] 1> "leader":"true"},
[junit4:junit4] 1> "5":{
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "node_name":"127.0.0.1:35433_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:35433/i_/o"}}},
[junit4:junit4] 1> "shard2":{
[junit4:junit4] 1> "range":"0-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{
[junit4:junit4] 1> "2":{
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "node_name":"127.0.0.1:57945_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:57945/i_/o",
[junit4:junit4] 1> "leader":"true"},
[junit4:junit4] 1> "4":{
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "node_name":"127.0.0.1:60439_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:60439/i_/o"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"1":{
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "node_name":"127.0.0.1:41482_i_%2Fo",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:41482/i_/o",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/live_nodes (5)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:57945_i_%2Fo (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:41482_i_%2Fo (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:35433_i_%2Fo (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:60439_i_%2Fo (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:40562_i_%2Fo (0)
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.381; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={_version_=-1432309728676937728&update.distrib=FROMLEADER&wt=javabin&version=2&update.from=http://127.0.0.1:57945/i_/o/collection1/} {deleteByQuery=*:* (-1432309728676937728)} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.384; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={_version_=-1432309728679034880&update.distrib=FROMLEADER&wt=javabin&version=2&update.from=http://127.0.0.1:40562/i_/o/collection1/} {deleteByQuery=*:* (-1432309728679034880)} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.385; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=TOLEADER&wt=javabin&version=2} {deleteByQuery=*:* (-1432309728679034880)} 0 5
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.385; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1432309728676937728)} 0 7
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.396; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[0 (1432309728686374912)]} 0 9
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.405; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:40562/i_/o/collection1/} {add=[0 (1432309728701054976)]} 0 2
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.406; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=TOLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:57945/i_/o/collection1/} {add=[0 (1432309728701054976)]} 0 6
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.406; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[0]} 0 8
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.409; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[1 (1432309728709443584)]} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.416; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:40562/i_/o/collection1/} {add=[1 (1432309728713637888)]} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.416; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=TOLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:57945/i_/o/collection1/} {add=[1 (1432309728713637888)]} 0 3
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.417; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[1]} 0 6
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.420; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[2 (1432309728719929344)]} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.429; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:57945/i_/o/collection1/} {add=[2 (1432309728725172224)]} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.430; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=TOLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:40562/i_/o/collection1/} {add=[2 (1432309728725172224)]} 0 6
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.431; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[2]} 0 9
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.433; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[3 (1432309728734609408)]} 0 0
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.440; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:57945/i_/o/collection1/} {add=[3 (1432309728738803712)]} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.441; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=TOLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:40562/i_/o/collection1/} {add=[3 (1432309728738803712)]} 0 4
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.441; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[3]} 0 6
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.444; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[4 (1432309728746143744)]} 0 0
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.449; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:40562/i_/o/collection1/} {add=[4 (1432309728748240896)]} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.449; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[4 (1432309728748240896)]} 0 3
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.452; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[5 (1432309728754532352)]} 0 0
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.458; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:57945/i_/o/collection1/} {add=[5 (1432309728757678080)]} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.459; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[5 (1432309728757678080)]} 0 4
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.461; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[6 (1432309728763969536)]} 0 0
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.468; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:57945/i_/o/collection1/} {add=[6 (1432309728769212416)]} 0 0
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.469; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=TOLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:40562/i_/o/collection1/} {add=[6 (1432309728769212416)]} 0 3
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.470; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[6]} 0 6
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.472; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[7 (1432309728775503872)]} 0 0
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.479; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:57945/i_/o/collection1/} {add=[7 (1432309728780746752)]} 0 0
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.480; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=TOLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:40562/i_/o/collection1/} {add=[7 (1432309728780746752)]} 0 3
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.480; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[7]} 0 5
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.483; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[8 (1432309728785989632)]} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.487; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:40562/i_/o/collection1/} {add=[8 (1432309728789135360)]} 0 0
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.488; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[8 (1432309728789135360)]} 0 3
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.490; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[9 (1432309728794378240)]} 0 0
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.498; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:57945/i_/o/collection1/} {add=[9 (1432309728799621120)]} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.499; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=TOLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:40562/i_/o/collection1/} {add=[9 (1432309728799621120)]} 0 4
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.500; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[9]} 0 8
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.503; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[10 (1432309728806961152)]} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.510; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:40562/i_/o/collection1/} {add=[10 (1432309728812204032)]} 0 1
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.511; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=TOLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:57945/i_/o/collection1/} {add=[10 (1432309728812204032)]} 0 4
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.511; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[10]} 0 6
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.514; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={wt=javabin&version=2} {add=[11 (1432309728819544064)]} 0 0
[junit4:junit4] 1> INFO - 2013-04-14 16:30:01.519; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp=/i_/o path=/update params={update.distrib=FROMLEADER&wt=javabin&version=2&distrib.from=http://127.0.0.1:40562/i_/o/collection1/} {add=[11 (1432309728822689792)]} 0 0
[junit4:junit4] 1> INFO - 2013-04-14 16:30:0
[...truncated too long message...]
ache.solr.update.DefaultSolrCoreState; SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.467; org.apache.solr.update.DefaultSolrCoreState; closing IndexWriter with IndexWriterCloser
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.468; org.apache.solr.core.SolrCore; [collection1] Closing main searcher on request.
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.469; org.apache.solr.core.CachingDirectoryFactory; Closing NRTCachingDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.469; org.apache.solr.core.CachingDirectoryFactory; looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4;done=false>>]
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.470; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.470; org.apache.solr.core.CachingDirectoryFactory; looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index;done=false>>]
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.470; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/org.apache.solr.cloud.ChaosMonkeyShardSplitTest-1365956975959/jetty4/index
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.471; org.apache.solr.cloud.Overseer$ClusterStateUpdater; According to ZK I (id=89519356376580108-127.0.0.1:35433_i_%2Fo-n_0000000004) am no longer a leader.
[junit4:junit4] 1> WARN - 2013-04-14 16:30:22.472; org.apache.solr.common.cloud.ZkStateReader$3; ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.472; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 1> WARN - 2013-04-14 16:30:22.473; org.apache.solr.common.cloud.ZkStateReader$2; ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.493; org.eclipse.jetty.server.handler.ContextHandler; stopped o.e.j.s.ServletContextHandler{/i_/o,null}
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.566; org.apache.solr.SolrTestCaseJ4; ###Ending testDistribSearch
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.567; org.apache.solr.cloud.ZkTestServer; connecting to 127.0.0.1:48131 48131
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J0/./solrtest-ChaosMonkeyShardSplitTest-1365956975959
[junit4:junit4] 2> check const of shard2
[junit4:junit4] 2> client0
[junit4:junit4] 2> PROPS:2:{"state":"active","shard":"shard2","node_name":"127.0.0.1:57945_i_%2Fo","core":"collection1","collection":"collection1","base_url":"http://127.0.0.1:57945/i_/o","leader":"true"}
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:48
[junit4:junit4] 2>
[junit4:junit4] 2> client1
[junit4:junit4] 2> PROPS:4:{"state":"active","shard":"shard2","node_name":"127.0.0.1:60439_i_%2Fo","core":"collection1","collection":"collection1","base_url":"http://127.0.0.1:60439/i_/o"}
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:47
[junit4:junit4] 2>
[junit4:junit4] 2> ######shard2 is not consistent. Got 48 from http://127.0.0.1:57945/i_/o/collection1lastClient and got 47 from http://127.0.0.1:60439/i_/o/collection1
[junit4:junit4] 2> ######http://127.0.0.1:57945/i_/o/collection1: SolrDocumentList[sz=48]=[SolrDocument{id=2, _version_=1432309728725172224}, SolrDocument{id=3, _version_=1432309728738803712}, SolrDocument{id=5, _version_=1432309728757678080}, SolrDocument{id=6, _version_=1432309728769212416}, SolrDocument{id=7, _version_=1432309728780746752}] , [...] , [SolrDocument{id=98, _version_=1432309729638481920}, SolrDocument{id=103, _version_=1432309734883459072}, SolrDocument{id=104, _version_=1432309735002996736}, SolrDocument{id=105, _version_=1432309735123582976}, SolrDocument{id=106, _version_=1432309735241023488}]
[junit4:junit4] 2> ######http://127.0.0.1:60439/i_/o/collection1: SolrDocumentList[sz=47]=[SolrDocument{id=2, _version_=1432309728725172224}, SolrDocument{id=3, _version_=1432309728738803712}, SolrDocument{id=5, _version_=1432309728757678080}, SolrDocument{id=6, _version_=1432309728769212416}, SolrDocument{id=7, _version_=1432309728780746752}] , [...] , [SolrDocument{id=97, _version_=1432309729629044736}, SolrDocument{id=98, _version_=1432309729638481920}, SolrDocument{id=103, _version_=1432309734883459072}, SolrDocument{id=104, _version_=1432309735002996736}, SolrDocument{id=105, _version_=1432309735123582976}]
[junit4:junit4] 2> ###### sizes=48,47
[junit4:junit4] 2> ###### Only in http://127.0.0.1:57945/i_/o/collection1: [{id=106, _version_=1432309735241023488}]
[junit4:junit4] 2> check const of shard1
[junit4:junit4] 2> client0
[junit4:junit4] 2> PROPS:3:{"state":"down","shard":"shard1","node_name":"127.0.0.1:40562_i_%2Fo","core":"collection1","collection":"collection1","base_url":"http://127.0.0.1:40562/i_/o"}
[junit4:junit4] 2> error contacting client: Server refused connection at: http://127.0.0.1:40562/i_/o/collection1
[junit4:junit4] 2>
[junit4:junit4] 2> client1
[junit4:junit4] 2> PROPS:5:{"state":"active","shard":"shard1","node_name":"127.0.0.1:35433_i_%2Fo","core":"collection1","collection":"collection1","base_url":"http://127.0.0.1:35433/i_/o","leader":"true"}
[junit4:junit4] 2> live:true
[junit4:junit4] 2> num:58
[junit4:junit4] 2>
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ChaosMonkeyShardSplitTest -Dtests.method=testDistribSearch -Dtests.seed=CCC8219427EC4A7A -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=sk_SK -Dtests.timezone=America/Nipigon -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 46.7s J0 | ChaosMonkeyShardSplitTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: shard2 is not consistent. Got 48 from http://127.0.0.1:57945/i_/o/collection1lastClient and got 47 from http://127.0.0.1:60439/i_/o/collection1
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([CCC8219427EC4A7A:4D2EAF8C50B32A46]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:963)
[junit4:junit4] > at org.apache.solr.cloud.ChaosMonkeyShardSplitTest.doTest(ChaosMonkeyShardSplitTest.java:188)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 1> INFO - 2013-04-14 16:30:22.608; org.apache.solr.SolrTestCaseJ4; ###deleteCore
[junit4:junit4] 2> 46656 T316 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 1> WARN - 2013-04-14 16:30:22.670; org.apache.solr.common.cloud.ZkStateReader$2; ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 1> WARN - 2013-04-14 16:30:22.670; org.apache.solr.common.cloud.ZkStateReader$3; ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=426), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=426)), sim=DefaultSimilarity, locale=sk_SK, timezone=America/Nipigon
[junit4:junit4] 2> NOTE: Linux 3.2.0-40-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=2,free=10779440,total=57933824
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestValueSourceCache, TestSolrIndexConfig, RecoveryZkTest, ZkCLITest, TestFieldCollectionResource, TestXIncludeConfig, PrimitiveFieldTypeTest, PeerSyncTest, TestSolrCoreProperties, ZkControllerTest, ChaosMonkeyShardSplitTest]
[junit4:junit4] Completed on J0 in 47.18s, 1 test, 1 failure <<< FAILURES!
[...truncated 807 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:375: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:355: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:438: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1231: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:875: There were test failures: 289 suites, 1201 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 35 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 32bit/jdk1.8.0-ea-b84 -client -XX:+UseSerialGC
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-ea-b84) - Build #
5168 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/5168/
Java: 32bit/jdk1.8.0-ea-b84 -server -XX:+UseParallelGC
2 tests failed.
REGRESSION: org.apache.solr.cloud.BasicDistributedZk2Test.testDistribSearch
Error Message:
Server at http://127.0.0.1:32918/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
Stack Trace:
org.apache.solr.common.SolrException: Server at http://127.0.0.1:32918/onenodecollectioncore returned non ok status:404, message:Can not find: /onenodecollectioncore/update
at __randomizedtesting.SeedInfo.seed([A767F9B17BD9D02D:268177A90C86B011]:0)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:375)
at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:116)
at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:102)
at org.apache.solr.cloud.BasicDistributedZk2Test.testNodeWithoutCollectionForwarding(BasicDistributedZk2Test.java:197)
at org.apache.solr.cloud.BasicDistributedZk2Test.doTest(BasicDistributedZk2Test.java:89)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
REGRESSION: org.apache.solr.cloud.UnloadDistributedZkTest.testDistribSearch
Error Message:
Still found shard2 in collection test_unload_shard_and_collection
Stack Trace:
java.lang.AssertionError: Still found shard2 in collection test_unload_shard_and_collection
at __randomizedtesting.SeedInfo.seed([A767F9B17BD9D02D:268177A90C86B011]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:127)
at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9188 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.BasicDistributedZk2Test
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.229; org.apache.solr.BaseDistributedSearchTestCase; Setting hostContext system property: /
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.232; org.apache.solr.SolrTestCaseJ4; ###Starting testDistribSearch
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.232; org.apache.solr.cloud.ZkTestServer; STARTING ZK TEST SERVER
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.233; org.apache.solr.cloud.ZkTestServer$ZKServerMain; Starting server
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.333; org.apache.solr.cloud.ZkTestServer; start zk server on port:56094
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.334; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.419; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1ef498c name:ZooKeeperConnection Watcher:127.0.0.1:56094 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.419; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.419; org.apache.solr.common.cloud.SolrZkClient; makePath: /solr
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.423; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.424; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@128b741 name:ZooKeeperConnection Watcher:127.0.0.1:56094/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.424; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.424; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.426; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/shards
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.428; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.429; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/shards
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.431; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.431; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.434; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.434; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.436; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.437; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.438; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.439; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/protwords.txt
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.440; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.441; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.443; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.443; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.445; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.445; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.447; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.448; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.449; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.450; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.511; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.529; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:48495
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.530; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.530; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.530; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1365968875453
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.531; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1365968875453/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.531; org.apache.solr.core.CoreContainer; New CoreContainer 10999733
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.531; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1365968875453/'
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.532; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1365968875453/'
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.549; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.550; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.550; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.550; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.551; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.551; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.551; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.552; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.552; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.553; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.559; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.567; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:56094/solr
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.568; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.569; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.570; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@2b4c1e name:ZooKeeperConnection Watcher:127.0.0.1:56094 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.570; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.572; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.575; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.576; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@19eb36a name:ZooKeeperConnection Watcher:127.0.0.1:56094/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.577; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.578; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/queue
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.580; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/collection-queue-work
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.582; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.583; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48495_
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.584; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:48495_
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.585; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer_elect/election
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.589; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer_elect/leader
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.590; org.apache.solr.cloud.Overseer; Overseer (id=89520136207400963-127.0.0.1:48495_-n_0000000000) starting
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.592; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/queue-work
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.594; org.apache.solr.cloud.OverseerCollectionProcessor; Process current queue of collection creations
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.594; org.apache.solr.common.cloud.SolrZkClient; makePath: /clusterstate.json
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.596; org.apache.solr.common.cloud.SolrZkClient; makePath: /aliases.json
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.597; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.598; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Starting to work on the main queue
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.600; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1365968875453/collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.600; org.apache.solr.cloud.ZkController; Check for collection zkNode:control_collection
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.601; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.601; org.apache.solr.cloud.ZkController; Load collection config from:/collections/control_collection
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.602; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1365968875453/collection1/'
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.602; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1365968875453/collection1/lib/README' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.602; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1365968875453/collection1/lib/classes/' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.623; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.649; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.650; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.653; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.937; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.941; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.943; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.952; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.955; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.957; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 19:47:55.958; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 19:47:55.958; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.958; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 19:47:55.959; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 19:47:55.959; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.959; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.959; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-controljetty-1365968875453/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/control/data/
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.960; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ae1d8
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.960; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.960; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/control/data
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.961; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/control/data/index/
[junit4:junit4] 1> WARN - 2013-04-14 19:47:55.961; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/control/data/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.961; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/control/data/index
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.962; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1406e08 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1bbb3c4),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.962; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.964; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.964; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.964; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.965; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.965; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.965; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.965; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.966; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.966; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.968; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.970; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@1f04ecf main
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.970; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/control/data/tlog
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.971; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.971; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.973; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@1f04ecf main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.973; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 19:47:55.973; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.101; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.102; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=1 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"1",
[junit4:junit4] 1> "shard":null,
[junit4:junit4] 1> "base_url":"http://127.0.0.1:48495",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:48495_",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "core_node_name":null,
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.102; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Create collection control_collection with numShards 1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.102; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.104; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.975; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.975; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:48495 collection:control_collection shard:shard1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.976; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/leader_elect/shard1/election
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.981; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.982; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.982; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.983; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:48495/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.983; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.983; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:48495/collection1/ has no replicas
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.983; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:48495/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 19:47:57.983; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/leaders/shard1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.607; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.614; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.639; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:48495/collection1/ and leader is http://127.0.0.1:48495/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.639; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:48495
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.639; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.639; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.639; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.641; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.642; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.642; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.642; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.645; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.646; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.647; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@a39b7b name:ZooKeeperConnection Watcher:127.0.0.1:56094/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.647; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.649; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.650; org.apache.solr.cloud.ChaosMonkey; monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.709; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.711; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:37964
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.711; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.712; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.712; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1365968878650
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.713; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1365968878650/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.713; org.apache.solr.core.CoreContainer; New CoreContainer 13640149
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.713; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1365968878650/'
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.714; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1365968878650/'
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.731; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.731; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.732; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.732; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.732; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.733; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.733; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.733; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.734; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.734; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.739; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.748; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:56094/solr
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.748; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.749; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.750; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1954a81 name:ZooKeeperConnection Watcher:127.0.0.1:56094 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.751; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.752; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.755; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.756; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@24d30a name:ZooKeeperConnection Watcher:127.0.0.1:56094/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.756; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:47:58.759; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.762; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37964_
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.763; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:37964_
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.768; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.768; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.769; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.768; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.776; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1365968878650/collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.776; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.777; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.778; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.779; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1365968878650/collection1/'
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.780; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1365968878650/collection1/lib/README' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.781; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1365968878650/collection1/lib/classes/' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.814; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.851; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.852; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:47:59.856; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.118; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.118; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=1 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"1",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:48495",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:48495_",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:48495__collection1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.121; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.121; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.121; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.177; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.182; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.184; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.193; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.195; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.197; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 19:48:00.198; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 19:48:00.199; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.199; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 19:48:00.199; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 19:48:00.200; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.200; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.200; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1365968878650/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty1/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.200; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ae1d8
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.201; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.201; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.201; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty1/index/
[junit4:junit4] 1> WARN - 2013-04-14 19:48:00.201; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty1/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.202; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty1/index
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.203; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@223ea7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68a7a9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.203; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.204; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.205; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.205; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.205; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.206; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.206; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.206; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.207; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.207; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.209; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.211; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@2d89b6 main
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.211; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty1/tlog
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.212; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.212; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.215; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@2d89b6 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.216; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 19:48:00.216; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:48:01.625; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:01.626; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard":null,
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37964",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:37964_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":null,
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:01.626; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Create collection collection1 with numShards 2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:01.626; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:01.632; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:01.632; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:01.632; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:02.217; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:02.218; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:37964 collection:collection1 shard:shard2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:02.218; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leader_elect/shard2/election
[junit4:junit4] 1> INFO - 2013-04-14 19:48:02.223; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:02.225; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:02.225; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4] 1> INFO - 2013-04-14 19:48:02.225; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:37964/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:02.226; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4] 1> INFO - 2013-04-14 19:48:02.226; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:37964/collection1/ has no replicas
[junit4:junit4] 1> INFO - 2013-04-14 19:48:02.226; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:37964/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:02.226; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leaders/shard2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.136; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.143; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.143; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.143; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.182; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:37964/collection1/ and leader is http://127.0.0.1:37964/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.183; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37964
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.183; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.183; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.184; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.185; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.187; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.187; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.187; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.249; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.252; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:32918
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.252; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.253; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.253; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1365968883190
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.253; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1365968883190/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.254; org.apache.solr.core.CoreContainer; New CoreContainer 1536057
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.254; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1365968883190/'
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.254; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1365968883190/'
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.271; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.272; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.272; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.272; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.273; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.273; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.273; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.274; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.274; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.274; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.279; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.288; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:56094/solr
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.288; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.289; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.290; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1f4162c name:ZooKeeperConnection Watcher:127.0.0.1:56094 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.291; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.293; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.296; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.297; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@c2c5e9 name:ZooKeeperConnection Watcher:127.0.0.1:56094/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.297; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:03.300; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.303; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32918_
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.304; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:32918_
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.306; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 2)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.306; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.307; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.307; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.307; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.307; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.308; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.311; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1365968883190/collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.311; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.312; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.312; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.313; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1365968883190/collection1/'
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.313; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1365968883190/collection1/lib/README' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.314; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1365968883190/collection1/lib/classes/' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.334; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.360; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.361; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.364; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.647; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.647; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37964",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:37964_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:37964__collection1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.649; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.650; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.650; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.650; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.650; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.654; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.656; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.665; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.668; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.670; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 19:48:04.671; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 19:48:04.672; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.672; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 19:48:04.673; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 19:48:04.673; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.673; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.673; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty2-1365968883190/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty2/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.673; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ae1d8
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.674; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.674; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.674; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty2/index/
[junit4:junit4] 1> WARN - 2013-04-14 19:48:04.675; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty2/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.675; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty2/index
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.676; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@631a86 lockFactory=org.apache.lucene.store.NativeFSLockFactory@fc50fc),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.676; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.677; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.678; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.678; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.679; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.679; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.679; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.679; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.680; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.680; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.682; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.684; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@d19ab9 main
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.684; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty2/tlog
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.685; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.685; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.687; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@d19ab9 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.689; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 19:48:04.689; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.153; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.154; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard":null,
[junit4:junit4] 1> "base_url":"http://127.0.0.1:32918",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:32918_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":null,
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.154; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Collection already exists with numShards=2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.154; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.159; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.159; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.159; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.159; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.690; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.690; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:32918 collection:collection1 shard:shard1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.691; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leader_elect/shard1/election
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.695; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.697; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.697; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.697; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:32918/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.697; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.698; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:32918/collection1/ has no replicas
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.698; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:32918/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:06.698; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/leaders/shard1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.663; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.668; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.668; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.668; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.668; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.704; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:32918/collection1/ and leader is http://127.0.0.1:32918/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.705; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:32918
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.705; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.705; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.705; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.706; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.707; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.708; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.708; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.768; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.778; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:52960
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.779; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.779; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.780; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1365968887711
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.780; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1365968887711/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.780; org.apache.solr.core.CoreContainer; New CoreContainer 18823354
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.781; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1365968887711/'
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.781; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1365968887711/'
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.799; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.800; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.800; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.800; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.801; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.801; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.801; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.801; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.802; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.802; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.807; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.816; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:56094/solr
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.817; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.817; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.819; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@b3dc3 name:ZooKeeperConnection Watcher:127.0.0.1:56094 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.819; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.821; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.824; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.825; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@eb0c20 name:ZooKeeperConnection Watcher:127.0.0.1:56094/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.825; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:07.830; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.832; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52960_
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.834; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:52960_
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.836; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.837; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 3)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.837; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.837; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.837; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.837; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.837; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.838; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.838; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.842; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1365968887711/collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.842; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.842; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.842; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.843; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1365968887711/collection1/'
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.844; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1365968887711/collection1/lib/README' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.844; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1365968887711/collection1/lib/classes/' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.865; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.892; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.893; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:08.896; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.171; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.172; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:32918",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:32918_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:32918__collection1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.175; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.175; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.175; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.175; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.175; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.193; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.198; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.200; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.210; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.212; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.215; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 19:48:09.215; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 19:48:09.216; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.216; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 19:48:09.217; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 19:48:09.217; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.217; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.217; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty3-1365968887711/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty3/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.218; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ae1d8
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.218; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.219; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty3
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.219; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty3/index/
[junit4:junit4] 1> WARN - 2013-04-14 19:48:09.219; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty3/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.219; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty3/index
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.220; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4e29b3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f75047),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.221; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.222; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.222; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.223; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.223; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.224; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.224; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.224; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.224; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.225; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.227; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.229; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@33abdb main
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.229; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty3/tlog
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.230; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.230; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.233; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@33abdb main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.234; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 19:48:09.234; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:48:10.679; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:10.679; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard":null,
[junit4:junit4] 1> "base_url":"http://127.0.0.1:52960",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:52960_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":null,
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:10.680; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Collection already exists with numShards=2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:10.680; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:10.682; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:10.682; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:10.682; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:10.682; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:10.683; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.235; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.236; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:52960 collection:collection1 shard:shard2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.238; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:52960/collection1/ and leader is http://127.0.0.1:37964/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.238; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52960
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.238; org.apache.solr.cloud.ZkController; Core needs to recover:collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.238; org.apache.solr.update.DefaultSolrCoreState; Running recovery - first canceling any ongoing recovery
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.239; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.239; org.apache.solr.cloud.RecoveryStrategy; Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.239; org.apache.solr.cloud.RecoveryStrategy; ###### startupVersions=[]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.240; org.apache.solr.cloud.ZkController; publishing core=collection1 state=recovering
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.240; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.240; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.240; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.241; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.241; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.245; org.apache.solr.handler.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:52960__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.307; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.309; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:37855
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.310; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.310; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.310; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1365968891245
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.311; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1365968891245/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.311; org.apache.solr.core.CoreContainer; New CoreContainer 11168934
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.311; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1365968891245/'
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.312; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1365968891245/'
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.330; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.331; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.331; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.332; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.332; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.332; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.333; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.333; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.333; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.334; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.339; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.348; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:56094/solr
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.349; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.350; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.351; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@b4829 name:ZooKeeperConnection Watcher:127.0.0.1:56094 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.351; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.353; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.356; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.357; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@2cc7 name:ZooKeeperConnection Watcher:127.0.0.1:56094/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.357; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:11.360; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.186; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.186; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:52960",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:52960_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:52960__collection1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"recovering",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.190; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.190; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.191; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.190; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.190; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.190; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.246; org.apache.solr.handler.admin.CoreAdminHandler; Waited coreNodeName: 127.0.0.1:52960__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.247; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={action=PREPRECOVERY&wt=javabin&version=2&checkLive=true&coreNodeName=127.0.0.1:52960__collection1&nodeName=127.0.0.1:52960_&onlyIfLeader=true&core=collection1&state=recovering} status=0 QTime=1002
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.362; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37855_
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.363; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:37855_
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.365; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.365; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.365; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.365; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.365; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.365; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.365; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.366; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.366; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.366; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.366; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.367; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.370; org.apache.solr.core.CoreContainer; Creating SolrCore 'collection1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1365968891245/collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.370; org.apache.solr.cloud.ZkController; Check for collection zkNode:collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.371; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.371; org.apache.solr.cloud.ZkController; Load collection config from:/collections/collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.372; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1365968891245/collection1/'
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.372; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1365968891245/collection1/lib/README' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.372; org.apache.solr.core.SolrResourceLoader; Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1365968891245/collection1/lib/classes/' to classloader
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.394; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.420; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.422; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.425; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.710; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.715; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.717; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.728; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.731; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.733; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 19:48:12.734; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 19:48:12.735; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.735; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 19:48:12.736; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 19:48:12.736; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.736; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.736; org.apache.solr.core.SolrCore; [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty4-1365968891245/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty4/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.736; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ae1d8
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.737; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.737; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty4
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.737; org.apache.solr.core.SolrCore; New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty4/index/
[junit4:junit4] 1> WARN - 2013-04-14 19:48:12.738; org.apache.solr.core.SolrCore; [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty4/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.738; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty4/index
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.739; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a515b5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e7c9d0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.739; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.741; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.741; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.741; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.742; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.742; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.742; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.742; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.743; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.743; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.745; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.747; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@15f38d4 main
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.747; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty4/tlog
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.748; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.748; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.750; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@15f38d4 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.752; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 19:48:12.752; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.694; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.695; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard":null,
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37855",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:37855_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":null,
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.695; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Collection already exists with numShards=2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.695; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.698; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.699; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.699; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.699; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.699; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.698; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.753; org.apache.solr.core.CoreContainer; registering core: collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.753; org.apache.solr.cloud.ZkController; Register replica - core:collection1 address:http://127.0.0.1:37855 collection:collection1 shard:shard1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.756; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:37855/collection1/ and leader is http://127.0.0.1:32918/collection1/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.756; org.apache.solr.cloud.ZkController; No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37855
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.756; org.apache.solr.cloud.ZkController; Core needs to recover:collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.756; org.apache.solr.update.DefaultSolrCoreState; Running recovery - first canceling any ongoing recovery
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.757; org.apache.solr.cloud.RecoveryStrategy; Starting recovery process. core=collection1 recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.757; org.apache.solr.cloud.RecoveryStrategy; ###### startupVersions=[]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.757; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.757; org.apache.solr.cloud.ZkController; publishing core=collection1 state=recovering
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.757; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.758; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.758; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.758; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.759; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.761; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.761; org.apache.solr.handler.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:37855__collection1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.762; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.765; org.apache.solr.core.CoreContainer; Creating SolrCore 'onenodecollectioncore' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1365968878650/onenodecollectioncore
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.765; org.apache.solr.cloud.ZkController; Check for collection zkNode:onenodecollection
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.766; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:onenodecollection
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.766; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.767; org.apache.solr.cloud.ZkController; Only one config set found in zk - using it:conf1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.768; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/onenodecollection
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.769; org.apache.solr.cloud.ZkController; Load collection config from:/collections/onenodecollection
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.770; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1365968878650/onenodecollectioncore/'
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.809; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.844; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.845; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:13.848; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.135; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.140; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.142; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.150; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.153; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.156; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 19:48:14.157; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 19:48:14.157; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.158; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 19:48:14.158; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 19:48:14.159; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.159; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.159; org.apache.solr.core.SolrCore; [onenodecollectioncore] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.BasicDistributedZk2Test-jetty1-1365968878650/onenodecollectioncore/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1365968875232/onenodecollection/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.160; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ae1d8
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.160; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.161; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1365968875232/onenodecollection
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.161; org.apache.solr.core.SolrCore; New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1365968875232/onenodecollection/index/
[junit4:junit4] 1> WARN - 2013-04-14 19:48:14.161; org.apache.solr.core.SolrCore; [onenodecollectioncore] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1365968875232/onenodecollection/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.162; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1365968875232/onenodecollection/index
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.163; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@cc36a5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18fa3),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.163; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.165; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.165; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.166; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.166; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.167; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.167; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.167; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.168; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.168; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.171; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.173; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@174dfdb main
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.173; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1365968875232/onenodecollection/tlog
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.174; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.174; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.177; org.apache.solr.core.SolrCore; [onenodecollectioncore] Registered new searcher Searcher@174dfdb main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.177; org.apache.solr.cloud.ZkController; publishing core=onenodecollectioncore state=down
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.247; org.apache.solr.cloud.RecoveryStrategy; Attempting to PeerSync from http://127.0.0.1:37964/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.247; org.apache.solr.update.PeerSync; PeerSync: core=collection1 url=http://127.0.0.1:52960 START replicas=[http://127.0.0.1:37964/collection1/] nUpdates=100
[junit4:junit4] 1> WARN - 2013-04-14 19:48:14.248; org.apache.solr.update.PeerSync; no frame of reference to tell of we've missed updates
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.248; org.apache.solr.cloud.RecoveryStrategy; PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.248; org.apache.solr.cloud.RecoveryStrategy; Starting Replication Recovery. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.248; org.apache.solr.cloud.RecoveryStrategy; Begin buffering updates. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.248; org.apache.solr.update.UpdateLog; Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.248; org.apache.solr.cloud.RecoveryStrategy; Attempting to replicate from http://127.0.0.1:37964/collection1/. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.249; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.251; org.apache.solr.core.SolrCore; [collection1] webapp= path=/get params={qt=/get&wt=javabin&version=2&getVersions=100&distrib=false} status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.252; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.253; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@223ea7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68a7a9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.253; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.254; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@223ea7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68a7a9),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@223ea7 lockFactory=org.apache.lucene.store.NativeFSLockFactory@68a7a9),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.254; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.254; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@154ad6e realtime
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.255; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.255; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp= path=/update params={softCommit=false&openSearcher=false&commit_end_point=true&wt=javabin&commit=true&version=2&waitSearcher=true} {commit=} 0 3
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.256; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for 10000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.256; org.apache.solr.handler.SnapPuller; No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.259; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for 10000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.260; org.apache.solr.core.SolrCore; [collection1] webapp= path=/replication params={qt=/replication&wt=javabin&command=indexversion&version=2} status=0 QTime=2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.261; org.apache.solr.handler.SnapPuller; Master's generation: 2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.261; org.apache.solr.handler.SnapPuller; Slave's generation: 1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.261; org.apache.solr.handler.SnapPuller; Starting replication process
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.263; org.apache.solr.core.SolrCore; [collection1] webapp= path=/replication params={qt=/replication&wt=javabin&command=filelist&version=2&generation=2} status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.263; org.apache.solr.handler.SnapPuller; Number of files in latest index in master: 1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.264; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty3/index.20130415064814263
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.264; org.apache.solr.handler.SnapPuller; Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@123fef5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@90c02d) fullCopy=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.266; org.apache.solr.core.SolrCore; [collection1] webapp= path=/replication params={qt=/replication&wt=filestream&command=filecontent&checksum=true&file=segments_2&generation=2} status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.267; org.apache.solr.handler.SnapPuller; Total time taken for download : 0 secs
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.267; org.apache.solr.update.DefaultSolrCoreState; Creating new IndexWriter...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.267; org.apache.solr.update.DefaultSolrCoreState; Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.268; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4e29b3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f75047),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@4e29b3 lockFactory=org.apache.lucene.store.NativeFSLockFactory@1f75047),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.269; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.269; org.apache.solr.update.DefaultSolrCoreState; New IndexWriter is ready to be used.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.269; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@f468a8 main
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.270; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@f468a8 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.270; org.apache.solr.core.CachingDirectoryFactory; looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty3/index.20130415064814263 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty3/index.20130415064814263;done=true>>]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.270; org.apache.solr.core.CachingDirectoryFactory; Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty3/index.20130415064814263
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.270; org.apache.solr.core.CachingDirectoryFactory; Removing directory before core close: ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty3/index.20130415064814263
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.270; org.apache.solr.cloud.RecoveryStrategy; No replay needed. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.271; org.apache.solr.cloud.RecoveryStrategy; Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.271; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.271; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:48:14.272; org.apache.solr.cloud.RecoveryStrategy; Finished recovery process. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.203; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.204; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37855",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:37855_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:37855__collection1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"recovering",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.207; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=1 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":"none",
[junit4:junit4] 1> "numShards":"1",
[junit4:junit4] 1> "shard":null,
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37964",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:37964_",
[junit4:junit4] 1> "collection":"onenodecollection",
[junit4:junit4] 1> "core_node_name":null,
[junit4:junit4] 1> "core":"onenodecollectioncore",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.207; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Create collection onenodecollection with numShards 1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.208; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Assigning new node to shard=shard1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.210; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:52960",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:52960_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:52960__collection1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.213; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.213; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.213; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.213; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.213; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.213; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.762; org.apache.solr.handler.admin.CoreAdminHandler; Waited coreNodeName: 127.0.0.1:37855__collection1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:15.763; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={action=PREPRECOVERY&wt=javabin&version=2&checkLive=true&coreNodeName=127.0.0.1:37855__collection1&nodeName=127.0.0.1:37855_&onlyIfLeader=true&core=collection1&state=recovering} status=0 QTime=2002
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.179; org.apache.solr.core.CoreContainer; registering core: onenodecollectioncore
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.179; org.apache.solr.cloud.ZkController; Register replica - core:onenodecollectioncore address:http://127.0.0.1:37964 collection:onenodecollection shard:shard1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.180; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/onenodecollection/leader_elect/shard1/election
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.184; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.186; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.186; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.186; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:37964/onenodecollectioncore/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.186; org.apache.solr.cloud.SyncStrategy; Sync Success - now sync replicas to me
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.187; org.apache.solr.cloud.SyncStrategy; http://127.0.0.1:37964/onenodecollectioncore/ has no replicas
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.187; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:37964/onenodecollectioncore/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.187; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/onenodecollection/leaders/shard1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.718; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.724; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.724; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.724; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.724; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.724; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.724; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.743; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:37964/onenodecollectioncore/ and leader is http://127.0.0.1:37964/onenodecollectioncore/
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.743; org.apache.solr.cloud.ZkController; No LogReplay needed for core=onenodecollectioncore baseURL=http://127.0.0.1:37964
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.743; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.744; org.apache.solr.cloud.ZkController; publishing core=onenodecollectioncore state=active
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.745; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.746; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={roles=none&action=CREATE&numShards=1&name=onenodecollectioncore&wt=javabin&version=2&dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-BasicDistributedZk2Test-1365968875232/onenodecollection&collection=onenodecollection} status=0 QTime=2981
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.747; org.apache.solr.cloud.AbstractDistribZkTestBase; Wait for recoveries to finish - collection: onenodecollection failOnTimeout:true timeout (sec):230
[junit4:junit4] 1> INFO - 2013-04-14 19:48:16.748; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.749; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.763; org.apache.solr.cloud.RecoveryStrategy; Attempting to PeerSync from http://127.0.0.1:32918/collection1/ core=collection1 - recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.764; org.apache.solr.update.PeerSync; PeerSync: core=collection1 url=http://127.0.0.1:37855 START replicas=[http://127.0.0.1:32918/collection1/] nUpdates=100
[junit4:junit4] 1> WARN - 2013-04-14 19:48:17.764; org.apache.solr.update.PeerSync; no frame of reference to tell of we've missed updates
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.764; org.apache.solr.cloud.RecoveryStrategy; PeerSync Recovery was not successful - trying replication. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.764; org.apache.solr.cloud.RecoveryStrategy; Starting Replication Recovery. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.764; org.apache.solr.cloud.RecoveryStrategy; Begin buffering updates. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.765; org.apache.solr.update.UpdateLog; Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.765; org.apache.solr.cloud.RecoveryStrategy; Attempting to replicate from http://127.0.0.1:32918/collection1/. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.765; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.768; org.apache.solr.core.SolrCore; [collection1] webapp= path=/get params={qt=/get&wt=javabin&version=2&getVersions=100&distrib=false} status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.768; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.769; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@631a86 lockFactory=org.apache.lucene.store.NativeFSLockFactory@fc50fc),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.769; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.770; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@631a86 lockFactory=org.apache.lucene.store.NativeFSLockFactory@fc50fc),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@631a86 lockFactory=org.apache.lucene.store.NativeFSLockFactory@fc50fc),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.770; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.770; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@480bfd realtime
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.771; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.771; org.apache.solr.update.processor.LogUpdateProcessor; [collection1] webapp= path=/update params={softCommit=false&openSearcher=false&commit_end_point=true&wt=javabin&commit=true&version=2&waitSearcher=true} {commit=} 0 3
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.771; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for 10000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.772; org.apache.solr.handler.SnapPuller; No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.773; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for 10000
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.773; org.apache.solr.core.SolrCore; [collection1] webapp= path=/replication params={qt=/replication&wt=javabin&command=indexversion&version=2} status=0 QTime=1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.774; org.apache.solr.handler.SnapPuller; Master's generation: 2
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.774; org.apache.solr.handler.SnapPuller; Slave's generation: 1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.774; org.apache.solr.handler.SnapPuller; Starting replication process
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.775; org.apache.solr.core.SolrCore; [collection1] webapp= path=/replication params={qt=/replication&wt=javabin&command=filelist&version=2&generation=2} status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.776; org.apache.solr.handler.SnapPuller; Number of files in latest index in master: 1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.776; org.apache.solr.core.CachingDirectoryFactory; return new directory for ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty4/index.20130415064817776
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.777; org.apache.solr.handler.SnapPuller; Starting download to MockDirWrapper(org.apache.lucene.store.RAMDirectory@3b0d5b lockFactory=org.apache.lucene.store.NativeFSLockFactory@e3d171) fullCopy=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.778; org.apache.solr.core.SolrCore; [collection1] webapp= path=/replication params={qt=/replication&wt=filestream&command=filecontent&checksum=true&file=segments_2&generation=2} status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.779; org.apache.solr.handler.SnapPuller; Total time taken for download : 0 secs
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.780; org.apache.solr.update.DefaultSolrCoreState; Creating new IndexWriter...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.780; org.apache.solr.update.DefaultSolrCoreState; Waiting until IndexWriter is unused... core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.781; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=2
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a515b5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e7c9d0),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@a515b5 lockFactory=org.apache.lucene.store.NativeFSLockFactory@e7c9d0),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.781; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.781; org.apache.solr.update.DefaultSolrCoreState; New IndexWriter is ready to be used.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.781; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@75d5d7 main
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.782; org.apache.solr.core.SolrCore; [collection1] Registered new searcher Searcher@75d5d7 main{StandardDirectoryReader(segments_2:1:nrt)}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.783; org.apache.solr.core.CachingDirectoryFactory; looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty4/index.20130415064817776 [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty4/index.20130415064817776;done=true>>]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.783; org.apache.solr.core.CachingDirectoryFactory; Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty4/index.20130415064817776
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.783; org.apache.solr.core.CachingDirectoryFactory; Removing directory before core close: ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/jetty4/index.20130415064817776
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.783; org.apache.solr.cloud.RecoveryStrategy; No replay needed. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.783; org.apache.solr.cloud.RecoveryStrategy; Replication Recovery was successful - registering as Active. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.783; org.apache.solr.cloud.ZkController; publishing core=collection1 state=active
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.783; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:48:17.785; org.apache.solr.cloud.RecoveryStrategy; Finished recovery process. core=collection1
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.228; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.229; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=1 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":"none",
[junit4:junit4] 1> "numShards":"1",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37964",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:37964_",
[junit4:junit4] 1> "collection":"onenodecollection",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:37964__onenodecollectioncore",
[junit4:junit4] 1> "core":"onenodecollectioncore",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.231; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37855",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:37855_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:37855__collection1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.233; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.233; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.234; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.234; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.233; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.233; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.751; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.752; org.apache.solr.cloud.AbstractDistribZkTestBase; Recoveries finished - collection: onenodecollection
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.752; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.761; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.762; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@b671e0 name:ZooKeeperConnection Watcher:127.0.0.1:56094 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.762; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> / (2)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /zookeeper (1)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr (7)
[junit4:junit4] 1> /solr/live_nodes (5)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:37964_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:32918_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:52960_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:48495_ (0)
[junit4:junit4] 1> /solr/live_nodes/127.0.0.1:37855_ (0)
[junit4:junit4] 1> /solr/collections (3)
[junit4:junit4] 1> /solr/collections/onenodecollection (2)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "configName":"conf1",
[junit4:junit4] 1> "router":"compositeId"}
[junit4:junit4] 1> /solr/collections/onenodecollection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/onenodecollection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/onenodecollection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/onenodecollection/leader_elect/shard1/election/89520136207400966-127.0.0.1:37964__onenodecollectioncore-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/onenodecollection/leaders (1)
[junit4:junit4] 1> /solr/collections/onenodecollection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37964",
[junit4:junit4] 1> "node_name":"127.0.0.1:37964_",
[junit4:junit4] 1> "core":"onenodecollectioncore"}
[junit4:junit4] 1> /solr/collections/control_collection (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/control_collection/shards (0)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election (1)
[junit4:junit4] 1> /solr/collections/control_collection/leader_elect/shard1/election/89520136207400963-127.0.0.1:48495__collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/control_collection/leaders (1)
[junit4:junit4] 1> /solr/collections/control_collection/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "base_url":"http://127.0.0.1:48495",
[junit4:junit4] 1> "node_name":"127.0.0.1:48495_",
[junit4:junit4] 1> "core":"collection1"}
[junit4:junit4] 1> /solr/collections/collection1 (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"configName":"conf1"}
[junit4:junit4] 1> /solr/collections/collection1/shards (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect (2)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard2 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard2/election (2)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/89520136207400970-127.0.0.1:52960__collection1-n_0000000001 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard2/election/89520136207400966-127.0.0.1:37964__collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1 (1)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election (2)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89520136207400968-127.0.0.1:32918__collection1-n_0000000000 (0)
[junit4:junit4] 1> /solr/collections/collection1/leader_elect/shard1/election/89520136207400972-127.0.0.1:37855__collection1-n_0000000001 (0)
[junit4:junit4] 1> /solr/collections/collection1/leaders (2)
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard2 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37964",
[junit4:junit4] 1> "node_name":"127.0.0.1:37964_",
[junit4:junit4] 1> "core":"collection1"}
[junit4:junit4] 1> /solr/collections/collection1/leaders/shard1 (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "base_url":"http://127.0.0.1:32918",
[junit4:junit4] 1> "node_name":"127.0.0.1:32918_",
[junit4:junit4] 1> "core":"collection1"}
[junit4:junit4] 1> /solr/overseer (3)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/overseer/queue (0)
[junit4:junit4] 1> /solr/overseer/queue-work (0)
[junit4:junit4] 1> /solr/overseer/collection-queue-work (0)
[junit4:junit4] 1> /solr/clusterstate.json (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "onenodecollection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"127.0.0.1:37964__onenodecollectioncore":{
[junit4:junit4] 1> "roles":"none",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37964",
[junit4:junit4] 1> "node_name":"127.0.0.1:37964_",
[junit4:junit4] 1> "collection":"onenodecollection",
[junit4:junit4] 1> "core":"onenodecollectioncore",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "collection1":{
[junit4:junit4] 1> "shards":{
[junit4:junit4] 1> "shard1":{
[junit4:junit4] 1> "range":"80000000-ffffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{
[junit4:junit4] 1> "127.0.0.1:32918__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:32918",
[junit4:junit4] 1> "node_name":"127.0.0.1:32918_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "leader":"true"},
[junit4:junit4] 1> "127.0.0.1:37855__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37855",
[junit4:junit4] 1> "node_name":"127.0.0.1:37855_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active"}}},
[junit4:junit4] 1> "shard2":{
[junit4:junit4] 1> "range":"0-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{
[junit4:junit4] 1> "127.0.0.1:37964__collection1":{
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37964",
[junit4:junit4] 1> "node_name":"127.0.0.1:37964_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "leader":"true"},
[junit4:junit4] 1> "127.0.0.1:52960__collection1":{
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:52960",
[junit4:junit4] 1> "node_name":"127.0.0.1:52960_",
[junit4:junit4] 1> "collection":"collection1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active"}}}},
[junit4:junit4] 1> "router":"compositeId"},
[junit4:junit4] 1> "control_collection":{
[junit4:junit4] 1> "shards":{"shard1":{
[junit4:junit4] 1> "range":"80000000-7fffffff",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "replicas":{"127.0.0.1:48495__collection1":{
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:48495",
[junit4:junit4] 1> "node_name":"127.0.0.1:48495_",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "leader":"true"}}}},
[junit4:junit4] 1> "router":"compositeId"}}
[junit4:junit4] 1> /solr/overseer_elect (2)
[junit4:junit4] 1> /solr/overseer_elect/leader (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {"id":"89520136207400963-127.0.0.1:48495_-n_0000000000"}
[junit4:junit4] 1> /solr/overseer_elect/election (5)
[junit4:junit4] 1> /solr/overseer_elect/election/89520136207400968-127.0.0.1:32918_-n_0000000002 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89520136207400966-127.0.0.1:37964_-n_0000000001 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89520136207400963-127.0.0.1:48495_-n_0000000000 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89520136207400972-127.0.0.1:37855_-n_0000000004 (0)
[junit4:junit4] 1> /solr/overseer_elect/election/89520136207400970-127.0.0.1:52960_-n_0000000003 (0)
[junit4:junit4] 1> /solr/configs (1)
[junit4:junit4] 1> /solr/configs/conf1 (9)
[junit4:junit4] 1> /solr/configs/conf1/old_synonyms.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/synonyms.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/mapping-ISOLatin1Accent.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/stopwords.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/schema.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/protwords.txt (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/solrconfig.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/currency.xml (0)
[junit4:junit4] 1> DATA: ...supressed...
[junit4:junit4] 1> /solr/configs/conf1/open-exchange-rates.json (0)
[junit4:junit4] 1> DATA:
[junit4:junit4] 1> {
[junit4:junit4] 1> "disclaimer": "This data is not real, it was synthetically created to match currency.xml. It is modeled after the data format available from openexchangerates.org. See https://openexchangerates.org/documentation for details",
[junit4:junit4] 1> "license": "http://www.apache.org/licenses/LICENSE-2.0",
[junit4:junit4] 1> "timestamp": 1332070464,
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> "IMPORTANT NOTE": "In order for tests to work, this data must be kept in sync with ./currency.xml",
[junit4:junit4] 1>
[junit4:junit4] 1>
[junit4:junit4] 1> "base": "USD",
[junit4:junit4] 1> "rates": {
[junit4:junit4] 1> "USD": 1,
[junit4:junit4] 1> "JPY": 81.29,
[junit4:junit4] 1> "EUR": 2.5,
[junit4:junit4] 1> "GBP": 0.5,
[junit4:junit4] 1> "MXN": 2.0
[junit4:junit4] 1> }
[junit4:junit4] 1> }
[junit4:junit4] 1>
[junit4:junit4] 1> /solr/aliases.json (0)
[junit4:junit4] 1>
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.792; org.apache.solr.cloud.ChaosMonkey; monkey: stop shard! 48495
[junit4:junit4] 1> INFO - 2013-04-14 19:48:18.794; org.apache.solr.core.CoreContainer; Shutting down CoreContainer instance=10999733
[junit4:junit4] 1> INFO - 2013-04-14 19:48:19.739; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:19.739; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=null message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:48495",
[junit4:junit4] 1> "node_name":"127.0.0.1:48495_",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:48495__collection1",
[junit4:junit4] 1> "core":"collection1",
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "operation":"state"}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:19.742; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:19.742; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:19.742; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:19.742; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:19.742; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.796; org.apache.solr.cloud.ZkController; publishing core=collection1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.796; org.apache.solr.cloud.ZkController; numShards not found on descriptor - reading it from system property
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.800; org.apache.solr.core.SolrCore; [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1b4d181
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.803; org.apache.solr.update.DirectUpdateHandler2; closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.804; org.apache.solr.update.SolrCoreState; Closing SolrCoreState
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.804; org.apache.solr.update.DefaultSolrCoreState; SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.804; org.apache.solr.update.DefaultSolrCoreState; closing IndexWriter with IndexWriterCloser
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.805; org.apache.solr.core.SolrCore; [collection1] Closing main searcher on request.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.805; org.apache.solr.core.CachingDirectoryFactory; Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.805; org.apache.solr.core.CachingDirectoryFactory; looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/control/data/index [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/control/data/index;done=false>>]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.806; org.apache.solr.core.CachingDirectoryFactory; Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/control/data/index
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.806; org.apache.solr.core.CachingDirectoryFactory; looking to close ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/control/data [CachedDir<<refCount=0;path=./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/control/data;done=false>>]
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.806; org.apache.solr.core.CachingDirectoryFactory; Closing directory: ./org.apache.solr.cloud.BasicDistributedZk2Test-1365968875232/control/data
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.807; org.apache.solr.cloud.Overseer$ClusterStateUpdater; According to ZK I (id=89520136207400963-127.0.0.1:48495_-n_0000000000) am no longer a leader.
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.809; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.809; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.809; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.809; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.810; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer_elect/leader
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.811; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.811; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.812; org.apache.solr.cloud.Overseer; Overseer (id=89520136207400966-127.0.0.1:37964_-n_0000000001) starting
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.814; org.apache.solr.cloud.OverseerCollectionProcessor; Process current queue of collection creations
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.815; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.815; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 4)
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.815; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Starting to work on the main queue
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.816; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 19:48:20.817; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:48495",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:48495_",
[junit4:junit4] 1> "collection":"control_collection",
[junit4:juni
[...truncated too long message...]
ler2; closing DirectUpdateHandler2{commits=3,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.875; org.apache.solr.update.SolrCoreState; Closing SolrCoreState
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.875; org.apache.solr.update.DefaultSolrCoreState; SolrCoreState ref count has reached 0 - closing IndexWriter
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.876; org.apache.solr.update.DefaultSolrCoreState; closing IndexWriter with IndexWriterCloser
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.877; org.apache.solr.core.SolrCore; [unloadcollection4] Closing main searcher on request.
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.877; org.apache.solr.core.CachingDirectoryFactory; Closing MockDirectoryFactory - 2 directories currently being tracked
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.877; org.apache.solr.core.CachingDirectoryFactory; looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-UnloadDistributedZkTest-1365968908941/1365968931075unloadcollection_4n [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-UnloadDistributedZkTest-1365968908941/1365968931075unloadcollection_4n;done=false>>]
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.878; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-UnloadDistributedZkTest-1365968908941/1365968931075unloadcollection_4n
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.878; org.apache.solr.core.CachingDirectoryFactory; looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-UnloadDistributedZkTest-1365968908941/1365968931075unloadcollection_4n/index.20130414144904581 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-UnloadDistributedZkTest-1365968908941/1365968931075unloadcollection_4n/index.20130414144904581;done=false>>]
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.878; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-UnloadDistributedZkTest-1365968908941/1365968931075unloadcollection_4n/index.20130414144904581
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.879; org.apache.solr.cloud.Overseer$ClusterStateUpdater; According to ZK I (id=89520138416553996-127.0.0.1:48420_ily-n_0000000004) am no longer a leader.
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.881; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (0)
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.881; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 0)
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.903; org.eclipse.jetty.server.handler.ContextHandler; stopped o.e.j.s.ServletContextHandler{/ily,null}
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.956; org.apache.solr.SolrTestCaseJ4; ###Ending testDistribSearch
[junit4:junit4] 1> INFO - 2013-04-14 19:50:46.957; org.apache.solr.cloud.ZkTestServer; connecting to 127.0.0.1:49243 49243
[junit4:junit4] 1> INFO - 2013-04-14 19:50:47.059; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1b33a5a name:ZooKeeperConnection Watcher:127.0.0.1:49243/solr got event WatchedEvent state:Disconnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 19:50:47.060; org.apache.solr.common.cloud.ConnectionManager; Client->ZooKeeper status change trigger but we are already closed
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-UnloadDistributedZkTest-1365968908941
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=UnloadDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=A767F9B17BD9D02D -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=de_LU -Dtests.timezone=Etc/GMT+5 -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 138s J1 | UnloadDistributedZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: Still found shard2 in collection test_unload_shard_and_collection
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([A767F9B17BD9D02D:268177A90C86B011]:0)
[junit4:junit4] > at org.apache.solr.cloud.UnloadDistributedZkTest.testUnloadShardAndCollection(UnloadDistributedZkTest.java:127)
[junit4:junit4] > at org.apache.solr.cloud.UnloadDistributedZkTest.doTest(UnloadDistributedZkTest.java:77)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 1> INFO - 2013-04-14 19:50:47.067; org.apache.solr.SolrTestCaseJ4; ###deleteCore
[junit4:junit4] 2> 138140 T2204 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 1> WARN - 2013-04-14 19:50:48.181; org.apache.solr.common.cloud.ZkStateReader$3; ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 1> INFO - 2013-04-14 19:50:48.181; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 1> WARN - 2013-04-14 19:50:48.181; org.apache.solr.common.cloud.ZkStateReader$2; ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 2> NOTE: test params are: codec=Lucene42: {a_si=Pulsing41(freqCutoff=14 minBlockSize=65 maxBlockSize=179), timestamp=Lucene41(blocksize=128), _version_=Pulsing41(freqCutoff=14 minBlockSize=65 maxBlockSize=179), range_facet_sl=PostingsFormat(name=SimpleText), text=PostingsFormat(name=Asserting), other_tl1=Pulsing41(freqCutoff=14 minBlockSize=65 maxBlockSize=179), range_facet_l=Lucene41(blocksize=128), a_t=Lucene41(blocksize=128), id=PostingsFormat(name=SimpleText), range_facet_si=Lucene41(blocksize=128), intDefault=Pulsing41(freqCutoff=14 minBlockSize=65 maxBlockSize=179), multiDefault=Lucene41(blocksize=128)}, docValues:{timestamp=DocValuesFormat(name=Lucene42)}, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=de_LU, timezone=Etc/GMT+5
[junit4:junit4] 2> NOTE: Linux 3.2.0-40-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=1,free=107005648,total=294518784
[junit4:junit4] 2> NOTE: All tests run in this JVM: [QueryResultKeyTest, SpellCheckComponentTest, DOMUtilTest, TestDistributedSearch, TestSchemaVersionResource, TestDefaultSearchFieldResource, TestSolr4Spatial, TestRandomFaceting, DocumentAnalysisRequestHandlerTest, TestAnalyzedSuggestions, TestStressReorder, CollectionsAPIDistributedZkTest, DistanceFunctionTest, DocValuesMultiTest, TimeZoneUtilsTest, SignatureUpdateProcessorFactoryTest, ShardRoutingCustomTest, StatsComponentTest, TestArbitraryIndexDir, DirectUpdateHandlerTest, IndexSchemaRuntimeFieldTest, TestBinaryResponseWriter, PingRequestHandlerTest, CurrencyFieldOpenExchangeTest, ZkNodePropsTest, NumericFieldsTest, TestPostingsSolrHighlighter, StandardRequestHandlerTest, TestAtomicUpdateErrorCases, DirectSolrSpellCheckerTest, SolrCoreCheckLockOnStartupTest, TestNumberUtils, TestCopyFieldCollectionResource, TestJmxIntegration, TestWordDelimiterFilterFactory, NoCacheHeaderTest, TestLuceneMatchVersion, HighlighterTest, TestCoreDiscovery, SpellCheckCollatorTest, TestQueryTypes, TestPartialUpdateDeduplication, TestIndexingPerformance, ChaosMonkeySafeLeaderTest, TestRTGBase, MultiTermTest, TestSolrQueryParserResource, FullSolrCloudDistribCmdsTest, TestSystemIdResolver, URLClassifyProcessorTest, SimplePostToolTest, TestSolrXMLSerializer, SpellPossibilityIteratorTest, NotRequiredUniqueKeyTest, TestZkChroot, DisMaxRequestHandlerTest, OpenCloseCoreStressTest, HardAutoCommitTest, TestSolrJ, TestUtils, TestSuggestSpellingConverter, UpdateParamsTest, SolrCmdDistributorTest, TestSchemaResource, TestElisionMultitermQuery, TestDefaultSimilarityFactory, TestClassNameShortening, OpenExchangeRatesOrgProviderTest, TestCSVLoader, BasicDistributedZk2Test, UnloadDistributedZkTest]
[junit4:junit4] Completed on J1 in 139.46s, 1 test, 1 failure <<< FAILURES!
[...truncated 441 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:375: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:355: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:438: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1231: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:875: There were test failures: 289 suites, 1201 tests, 1 error, 1 failure, 13 ignored (7 assumptions)
Total time: 37 minutes 38 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 32bit/jdk1.8.0-ea-b84 -server -XX:+UseParallelGC
Email was triggered for: Failure
Sending email for trigger: Failure
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.8.0-ea-b84) - Build #
5167 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/5167/
Java: 32bit/jdk1.8.0-ea-b84 -server -XX:+UseG1GC
1 tests failed.
REGRESSION: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch
Error Message:
There are still nodes recoverying - waited for 230 seconds
Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
at __randomizedtesting.SeedInfo.seed([DDBBB8F1126DC87F:5C5D36E96532A843]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:483)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:487)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
at java.lang.Thread.run(Thread.java:722)
Build Log:
[...truncated 9300 lines...]
[junit4:junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.665; org.apache.solr.BaseDistributedSearchTestCase; Setting hostContext system property: /
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.669; org.apache.solr.SolrTestCaseJ4; ###Starting testDistribSearch
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.670; org.apache.solr.cloud.ZkTestServer; STARTING ZK TEST SERVER
[junit4:junit4] 1> client port:0.0.0.0/0.0.0.0:0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.671; org.apache.solr.cloud.ZkTestServer$ZKServerMain; Starting server
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.771; org.apache.solr.cloud.ZkTestServer; start zk server on port:52162
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.773; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.967; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@e63d87 name:ZooKeeperConnection Watcher:127.0.0.1:52162 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.967; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.968; org.apache.solr.common.cloud.SolrZkClient; makePath: /solr
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.972; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.973; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@21ccab name:ZooKeeperConnection Watcher:127.0.0.1:52162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.973; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.973; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.976; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/collection1/shards
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.977; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.979; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/control_collection/shards
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.980; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.981; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.984; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.984; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.987; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.987; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/stopwords.txt
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.989; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.989; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/protwords.txt
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.991; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.991; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.993; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.994; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/open-exchange-rates.json
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.996; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.996; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.998; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4:junit4] 1> INFO - 2013-04-14 18:27:42.998; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/old_synonyms.txt
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.000; org.apache.solr.cloud.AbstractZkTestCase; put /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.001; org.apache.solr.common.cloud.SolrZkClient; makePath: /configs/conf1/synonyms.txt
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.089; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.095; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:32786
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.096; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.096; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.097; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.097; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.098; org.apache.solr.core.CoreContainer; New CoreContainer 21774330
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.098; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.098; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.123; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.123; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.124; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.124; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.124; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.125; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.125; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.125; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.126; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.126; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.141; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.152; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:52162/solr
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.152; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.153; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.154; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1cc624c name:ZooKeeperConnection Watcher:127.0.0.1:52162 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.155; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.157; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.159; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.160; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@22d582 name:ZooKeeperConnection Watcher:127.0.0.1:52162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.161; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.162; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/queue
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.165; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/collection-queue-work
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.167; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.168; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32786_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.169; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:32786_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.171; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer_elect/election
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.175; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer_elect/leader
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.177; org.apache.solr.cloud.Overseer; Overseer (id=89519820811272195-127.0.0.1:32786_-n_0000000000) starting
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.179; org.apache.solr.common.cloud.SolrZkClient; makePath: /overseer/queue-work
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.181; org.apache.solr.cloud.OverseerCollectionProcessor; Process current queue of collection creations
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.182; org.apache.solr.common.cloud.SolrZkClient; makePath: /clusterstate.json
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.183; org.apache.solr.common.cloud.SolrZkClient; makePath: /aliases.json
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.185; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.186; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Starting to work on the main queue
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.188; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.188; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.189; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.191; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.192; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.193; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@4a5066 name:ZooKeeperConnection Watcher:127.0.0.1:52162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.193; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.194; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.196; org.apache.solr.cloud.ChaosMonkey; monkey: init - expire sessions:true cause connection loss:true
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.257; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.259; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:44653
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.259; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.260; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.260; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.260; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.261; org.apache.solr.core.CoreContainer; New CoreContainer 28902672
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.261; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.261; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.280; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.280; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.281; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.281; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.281; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.282; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.282; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.282; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.283; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.283; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.289; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.298; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:52162/solr
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.299; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.300; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.301; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@ca0cc4 name:ZooKeeperConnection Watcher:127.0.0.1:52162 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.301; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.302; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.305; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.307; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@13437a name:ZooKeeperConnection Watcher:127.0.0.1:52162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.308; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:43.312; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.314; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44653_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.315; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:44653_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.318; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.318; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.318; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (2)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.322; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.323; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.323; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.398; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.400; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:46976
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.401; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.401; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.401; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.402; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.402; org.apache.solr.core.CoreContainer; New CoreContainer 27955457
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.403; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.403; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.422; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.423; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.423; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.424; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.424; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.425; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.425; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.425; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.426; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.426; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.433; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.443; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:52162/solr
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.444; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.444; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.446; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1e8ffb9 name:ZooKeeperConnection Watcher:127.0.0.1:52162 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.446; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.448; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.451; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.452; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@46d44f name:ZooKeeperConnection Watcher:127.0.0.1:52162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.452; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:44.455; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.457; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46976_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.458; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:46976_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.460; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.461; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.460; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.461; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (3)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.464; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.464; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.465; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.542; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.544; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:37645
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.545; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.545; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.546; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.546; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.546; org.apache.solr.core.CoreContainer; New CoreContainer 12322176
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.547; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.547; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.566; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.567; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.567; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.567; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.568; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.568; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.569; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.569; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.569; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.570; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.575; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.585; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:52162/solr
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.586; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.586; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.588; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1fbbe25 name:ZooKeeperConnection Watcher:127.0.0.1:52162 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.588; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.590; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.592; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.593; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@16cfcf1 name:ZooKeeperConnection Watcher:127.0.0.1:52162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.594; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:45.597; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.599; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37645_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.601; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:37645_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.603; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.604; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.604; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.604; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.604; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (4)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.608; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.609; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.609; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.691; org.eclipse.jetty.server.Server; jetty-8.1.8.v20121106
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.693; org.eclipse.jetty.server.AbstractConnector; Started SelectChannelConnector@127.0.0.1:51793
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.693; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init()
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.694; org.apache.solr.core.SolrResourceLoader; JNDI not configured for solr (NoInitialContextEx)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.694; org.apache.solr.core.SolrResourceLoader; using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.695; org.apache.solr.core.CoreContainer$Initializer; looking for solr config file: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.695; org.apache.solr.core.CoreContainer; New CoreContainer 17855097
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.696; org.apache.solr.core.CoreContainer; Loading CoreContainer using Solr Home: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.696; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.717; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting socketTimeout to: 120000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.717; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting urlScheme to: http://
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.718; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting connTimeout to: 15000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.718; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxConnectionsPerHost to: 20
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.719; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting corePoolSize to: 0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.719; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maximumPoolSize to: 2147483647
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.719; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting maxThreadIdleTime to: 5
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.720; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting sizeOfQueue to: -1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.720; org.apache.solr.handler.component.HttpShardHandlerFactory; Setting fairnessPolicy to: false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.721; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnectionsPerHost=20&maxConnections=10000&socketTimeout=120000&connTimeout=15000&retry=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.727; org.apache.solr.core.CoreContainer; Registering Log Listener
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.738; org.apache.solr.core.CoreContainer; Zookeeper client=127.0.0.1:52162/solr
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.738; org.apache.solr.cloud.ZkController; zkHost includes chroot
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.739; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.740; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@5ed1f7 name:ZooKeeperConnection Watcher:127.0.0.1:52162 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.741; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.742; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=500&maxConnectionsPerHost=16&socketTimeout=120000&connTimeout=15000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.745; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.746; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@c0aa3f name:ZooKeeperConnection Watcher:127.0.0.1:52162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.746; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:46.749; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.751; org.apache.solr.cloud.ZkController; Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51793_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.753; org.apache.solr.common.cloud.SolrZkClient; makePath: /live_nodes/127.0.0.1:51793_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.755; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.755; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.756; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.756; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.756; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.756; org.apache.solr.common.cloud.ZkStateReader$3; Updating live nodes... (5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.760; org.apache.solr.servlet.SolrDispatchFilter; user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.760; org.apache.solr.servlet.SolrDispatchFilter; SolrDispatchFilter.init() done
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.761; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.765; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.771; org.apache.solr.handler.admin.CollectionsHandler; Creating Collection : replicationFactor=2&wt=javabin&version=2&action=CREATE&numShards=2&name=nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.772; org.apache.solr.cloud.DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.773; org.apache.solr.cloud.OverseerCollectionProcessor; Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4:junit4] 1> "replicationFactor":"2",
[junit4:junit4] 1> "maxShardsPerNode":null,
[junit4:junit4] 1> "operation":"createcollection",
[junit4:junit4] 1> "createNodeSet":null,
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "name":"nodes_used_collection"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.773; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard nodes_used_collection_shard1_replica1 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:51793_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.774; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard nodes_used_collection_shard1_replica2 as part of slice shard1 of collection nodes_used_collection on 127.0.0.1:44653_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.774; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard nodes_used_collection_shard2_replica1 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:46976_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.775; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard nodes_used_collection_shard2_replica2 as part of slice shard2 of collection nodes_used_collection on 127.0.0.1:32786_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.775; org.apache.solr.core.CoreContainer; Creating SolrCore 'nodes_used_collection_shard1_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/nodes_used_collection_shard1_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.776; org.apache.solr.cloud.ZkController; Check for collection zkNode:nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.776; org.apache.solr.core.CoreContainer; Creating SolrCore 'nodes_used_collection_shard2_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/nodes_used_collection_shard2_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.776; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.775; org.apache.solr.core.CoreContainer; Creating SolrCore 'nodes_used_collection_shard1_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/nodes_used_collection_shard1_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.776; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.776; org.apache.solr.cloud.ZkController; Check for collection zkNode:nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.776; org.apache.solr.core.CoreContainer; Creating SolrCore 'nodes_used_collection_shard2_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/nodes_used_collection_shard2_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.778; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.778; org.apache.solr.cloud.ZkController; Only one config set found in zk - using it:conf1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.777; org.apache.solr.cloud.ZkController; Check for collection zkNode:nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.779; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.779; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.780; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.778; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.778; org.apache.solr.cloud.ZkController; Check for collection zkNode:nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.781; org.apache.solr.cloud.ZkController; Only one config set found in zk - using it:conf1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.781; org.apache.solr.cloud.ZkController; Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.781; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.782; org.apache.solr.cloud.ZkController; Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.782; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.781; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.783; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/nodes_used_collection_shard1_replica2/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.783; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/nodes_used_collection_shard2_replica1/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.784; org.apache.solr.cloud.ZkController; Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.785; org.apache.solr.cloud.ZkController; Load collection config from:/collections/nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.786; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/nodes_used_collection_shard2_replica2/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.787; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/nodes_used_collection_shard1_replica1/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.812; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.822; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.825; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.839; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.860; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.862; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.866; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.869; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.870; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.874; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.888; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.890; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.892; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.893; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.896; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:47.900; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.334; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.340; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.343; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.354; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.355; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.357; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.360; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.361; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.362; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.362; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.363; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.363; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.364; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.364; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.364; org.apache.solr.core.SolrCore; [nodes_used_collection_shard2_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/nodes_used_collection_shard2_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/nodes_used_collection_shard2_replica2/data/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.365; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13918b0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.365; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.366; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/nodes_used_collection_shard2_replica2/data
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.366; org.apache.solr.core.SolrCore; New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/nodes_used_collection_shard2_replica2/data/index/
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.367; org.apache.solr.core.SolrCore; [nodes_used_collection_shard2_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/nodes_used_collection_shard2_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.367; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/nodes_used_collection_shard2_replica2/data/index
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.368; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.369; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fcd4f1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18744cc),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.369; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.371; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.372; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.373; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.373; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.374; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.374; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.375; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.375; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.376; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.379; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.379; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.381; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.381; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@6b8968 main
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.381; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/nodes_used_collection_shard2_replica2/data/tlog
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.382; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.383; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.394; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.396; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.399; org.apache.solr.cloud.ZkController; publishing core=nodes_used_collection_shard2_replica2 state=down
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.399; org.apache.solr.core.SolrCore; [nodes_used_collection_shard2_replica2] Registered new searcher Searcher@6b8968 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.399; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.402; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.403; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.405; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.404; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.405; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.406; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.406; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.407; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.407; org.apache.solr.core.SolrCore; [nodes_used_collection_shard1_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/nodes_used_collection_shard1_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/nodes_used_collection_shard1_replica1/data/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.408; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13918b0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.408; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.409; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/nodes_used_collection_shard1_replica1/data
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.409; org.apache.solr.core.SolrCore; New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/nodes_used_collection_shard1_replica1/data/index/
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.409; org.apache.solr.core.SolrCore; [nodes_used_collection_shard1_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/nodes_used_collection_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.410; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.410; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/nodes_used_collection_shard1_replica1/data/index
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.412; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.412; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.412; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@82848a lockFactory=org.apache.lucene.store.NativeFSLockFactory@7668e2),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.413; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.417; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.417; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.418; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.419; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.420; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.420; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.421; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.421; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.422; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.422; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.424; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.425; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.425; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.425; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.425; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.427; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.427; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.428; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.428; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@d1e305 main
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.428; org.apache.solr.core.SolrCore; [nodes_used_collection_shard1_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/nodes_used_collection_shard1_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/nodes_used_collection_shard1_replica2/data/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.428; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/nodes_used_collection_shard1_replica1/data/tlog
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.429; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13918b0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.429; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.429; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.430; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.430; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.431; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/nodes_used_collection_shard1_replica2/data
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.432; org.apache.solr.core.SolrCore; New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/nodes_used_collection_shard1_replica2/data/index/
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.432; org.apache.solr.core.SolrCore; [nodes_used_collection_shard1_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/nodes_used_collection_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.433; org.apache.solr.core.SolrCore; [nodes_used_collection_shard1_replica1] Registered new searcher Searcher@d1e305 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.433; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/nodes_used_collection_shard1_replica2/data/index
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.434; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.434; org.apache.solr.cloud.ZkController; publishing core=nodes_used_collection_shard1_replica1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.436; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@62770f lockFactory=org.apache.lucene.store.NativeFSLockFactory@1cc0135),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.436; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.438; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.438; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.438; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.439; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.439; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.439; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.440; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.440; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.440; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.441; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.441; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.441; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.442; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.442; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.442; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.443; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.443; org.apache.solr.core.SolrCore; [nodes_used_collection_shard2_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/nodes_used_collection_shard2_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/nodes_used_collection_shard2_replica1/data/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.443; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13918b0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.444; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.444; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/nodes_used_collection_shard2_replica1/data
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.445; org.apache.solr.core.SolrCore; New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/nodes_used_collection_shard2_replica1/data/index/
[junit4:junit4] 1> WARN - 2013-04-14 18:27:48.445; org.apache.solr.core.SolrCore; [nodes_used_collection_shard2_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/nodes_used_collection_shard2_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.445; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.446; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/nodes_used_collection_shard2_replica1/data/index
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.447; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@2fed7b lockFactory=org.apache.lucene.store.NativeFSLockFactory@1d953ff),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.447; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@1e3536b main
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.448; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.448; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/nodes_used_collection_shard1_replica2/data/tlog
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.449; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.449; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.450; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.450; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.451; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.451; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.452; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.452; org.apache.solr.core.SolrCore; [nodes_used_collection_shard1_replica2] Registered new searcher Searcher@1e3536b main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.452; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.452; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.453; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.453; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.453; org.apache.solr.cloud.ZkController; publishing core=nodes_used_collection_shard1_replica2 state=down
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.456; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.458; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@15f3b34 main
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.459; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/nodes_used_collection_shard2_replica1/data/tlog
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.459; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.459; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.462; org.apache.solr.core.SolrCore; [nodes_used_collection_shard2_replica1] Registered new searcher Searcher@15f3b34 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:48.464; org.apache.solr.cloud.ZkController; publishing core=nodes_used_collection_shard2_replica1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.194; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.194; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "collection":"nodes_used_collection",
[junit4:junit4] 1> "core_node_name":null,
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:32786_",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:32786"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.195; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Create collection nodes_used_collection with numShards 2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.197; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "collection":"nodes_used_collection",
[junit4:junit4] 1> "core_node_name":null,
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:51793_",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:51793"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.200; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "collection":"nodes_used_collection",
[junit4:junit4] 1> "core_node_name":null,
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:44653_",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:44653"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.202; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "collection":"nodes_used_collection",
[junit4:junit4] 1> "core_node_name":null,
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:46976_",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:46976"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.209; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.209; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.209; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.209; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.209; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.210; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.401; org.apache.solr.core.CoreContainer; registering core: nodes_used_collection_shard2_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.401; org.apache.solr.cloud.ZkController; Register replica - core:nodes_used_collection_shard2_replica2 address:http://127.0.0.1:32786 collection:nodes_used_collection shard:shard2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.402; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/nodes_used_collection/leader_elect/shard2/election
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.406; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.408; org.apache.solr.cloud.ShardLeaderElectionContext; Waiting until we see more replicas up: total=2 found=1 timeoutin=180000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.435; org.apache.solr.core.CoreContainer; registering core: nodes_used_collection_shard1_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.436; org.apache.solr.cloud.ZkController; Register replica - core:nodes_used_collection_shard1_replica1 address:http://127.0.0.1:51793 collection:nodes_used_collection shard:shard1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.438; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/nodes_used_collection/leader_elect/shard1/election
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.444; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.446; org.apache.solr.cloud.ShardLeaderElectionContext; Waiting until we see more replicas up: total=2 found=1 timeoutin=180000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.455; org.apache.solr.core.CoreContainer; registering core: nodes_used_collection_shard1_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.455; org.apache.solr.cloud.ZkController; Register replica - core:nodes_used_collection_shard1_replica2 address:http://127.0.0.1:44653 collection:nodes_used_collection shard:shard1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.465; org.apache.solr.core.CoreContainer; registering core: nodes_used_collection_shard2_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.465; org.apache.solr.cloud.ZkController; Register replica - core:nodes_used_collection_shard2_replica1 address:http://127.0.0.1:46976 collection:nodes_used_collection shard:shard2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.909; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.909; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.909; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:32786/nodes_used_collection_shard2_replica2/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.910; org.apache.solr.update.PeerSync; PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:32786 START replicas=[http://127.0.0.1:46976/nodes_used_collection_shard2_replica1/] nUpdates=100
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.910; org.apache.solr.update.PeerSync; PeerSync: core=nodes_used_collection_shard2_replica2 url=http://127.0.0.1:32786 DONE. We have no versions. sync failed.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.911; org.apache.solr.cloud.SyncStrategy; Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.911; org.apache.solr.cloud.ShardLeaderElectionContext; We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.911; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:32786/nodes_used_collection_shard2_replica2/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.912; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/nodes_used_collection/leaders/shard2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.915; org.apache.solr.core.SolrCore; [nodes_used_collection_shard2_replica1] webapp= path=/get params={wt=javabin&version=2&getVersions=100&distrib=false&qt=/get} status=0 QTime=1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.947; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.948; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.949; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:51793/nodes_used_collection_shard1_replica1/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.950; org.apache.solr.update.PeerSync; PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:51793 START replicas=[http://127.0.0.1:44653/nodes_used_collection_shard1_replica2/] nUpdates=100
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.950; org.apache.solr.update.PeerSync; PeerSync: core=nodes_used_collection_shard1_replica1 url=http://127.0.0.1:51793 DONE. We have no versions. sync failed.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.951; org.apache.solr.cloud.SyncStrategy; Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.952; org.apache.solr.cloud.ShardLeaderElectionContext; We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.952; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:51793/nodes_used_collection_shard1_replica1/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.953; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/nodes_used_collection/leaders/shard1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:49.958; org.apache.solr.core.SolrCore; [nodes_used_collection_shard1_replica2] webapp= path=/get params={wt=javabin&version=2&getVersions=100&distrib=false&qt=/get} status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.717; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.727; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.727; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.727; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.727; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.727; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.727; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.758; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:44653/nodes_used_collection_shard1_replica2/ and leader is http://127.0.0.1:51793/nodes_used_collection_shard1_replica1/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.759; org.apache.solr.cloud.ZkController; No LogReplay needed for core=nodes_used_collection_shard1_replica2 baseURL=http://127.0.0.1:44653
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.760; org.apache.solr.cloud.ZkController; Core needs to recover:nodes_used_collection_shard1_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.760; org.apache.solr.update.DefaultSolrCoreState; Running recovery - first canceling any ongoing recovery
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.761; org.apache.solr.cloud.RecoveryStrategy; Starting recovery process. core=nodes_used_collection_shard1_replica2 recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.761; org.apache.solr.cloud.RecoveryStrategy; ###### startupVersions=[]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.762; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.762; org.apache.solr.cloud.ZkController; publishing core=nodes_used_collection_shard1_replica2 state=recovering
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.763; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:51793/nodes_used_collection_shard1_replica1/ and leader is http://127.0.0.1:51793/nodes_used_collection_shard1_replica1/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.763; org.apache.solr.core.CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.763; org.apache.solr.cloud.ZkController; No LogReplay needed for core=nodes_used_collection_shard1_replica1 baseURL=http://127.0.0.1:51793
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.764; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.764; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.764; org.apache.solr.cloud.ZkController; publishing core=nodes_used_collection_shard1_replica1 state=active
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.766; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.767; org.apache.solr.core.CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.768; org.apache.solr.handler.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:44653__nodes_used_collection_shard1_replica2, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.768; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:46976/nodes_used_collection_shard2_replica1/ and leader is http://127.0.0.1:32786/nodes_used_collection_shard2_replica2/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.768; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:32786/nodes_used_collection_shard2_replica2/ and leader is http://127.0.0.1:32786/nodes_used_collection_shard2_replica2/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.769; org.apache.solr.cloud.ZkController; No LogReplay needed for core=nodes_used_collection_shard2_replica2 baseURL=http://127.0.0.1:32786
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.769; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.769; org.apache.solr.cloud.ZkController; No LogReplay needed for core=nodes_used_collection_shard2_replica1 baseURL=http://127.0.0.1:46976
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.770; org.apache.solr.cloud.ZkController; publishing core=nodes_used_collection_shard2_replica2 state=active
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.770; org.apache.solr.cloud.ZkController; Core needs to recover:nodes_used_collection_shard2_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.770; org.apache.solr.core.SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.771; org.apache.solr.update.DefaultSolrCoreState; Running recovery - first canceling any ongoing recovery
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.771; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={collection=nodes_used_collection&shard=shard1&wt=javabin&version=2&action=CREATE&numShards=2&qt=/admin/cores&name=nodes_used_collection_shard1_replica2} status=0 QTime=2996
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.772; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.772; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.772; org.apache.solr.cloud.RecoveryStrategy; Starting recovery process. core=nodes_used_collection_shard2_replica1 recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.773; org.apache.solr.core.CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.773; org.apache.solr.core.CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.773; org.apache.solr.cloud.RecoveryStrategy; ###### startupVersions=[]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.774; org.apache.solr.cloud.ZkController; publishing core=nodes_used_collection_shard2_replica1 state=recovering
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.774; org.apache.solr.core.SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.776; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={collection=nodes_used_collection&shard=shard1&wt=javabin&version=2&action=CREATE&numShards=2&qt=/admin/cores&name=nodes_used_collection_shard1_replica1} status=0 QTime=3001
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.778; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.781; org.apache.solr.handler.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:46976__nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.782; org.apache.solr.core.SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.783; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={collection=nodes_used_collection&shard=shard2&wt=javabin&version=2&action=CREATE&numShards=2&qt=/admin/cores&name=nodes_used_collection_shard2_replica2} status=0 QTime=3007
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.783; org.apache.solr.core.SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.784; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={collection=nodes_used_collection&shard=shard2&wt=javabin&version=2&action=CREATE&numShards=2&qt=/admin/cores&name=nodes_used_collection_shard2_replica1} status=0 QTime=3008
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.785; org.apache.solr.cloud.OverseerCollectionProcessor; Finished create command on all shards for collection: nodes_used_collection
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.827; org.apache.solr.cloud.DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/collection-queue-work/qnr-0000000000 state: SyncConnected type NodeDataChanged
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.828; org.apache.solr.cloud.DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.829; org.apache.solr.cloud.OverseerCollectionProcessor; Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000000 complete, response:{success={null={responseHeader={status=0,QTime=2996},core=nodes_used_collection_shard1_replica2,saved=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/solr.xml},null={responseHeader={status=0,QTime=3001},core=nodes_used_collection_shard1_replica1,saved=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/solr.xml},null={responseHeader={status=0,QTime=3007},core=nodes_used_collection_shard2_replica2,saved=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/solr.xml},null={responseHeader={status=0,QTime=3008},core=nodes_used_collection_shard2_replica1,saved=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/solr.xml}}}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.830; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/collections params={replicationFactor=2&wt=javabin&version=2&action=CREATE&numShards=2&name=nodes_used_collection} status=0 QTime=3061
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.831; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.831; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.833; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@1b08334 name:ZooKeeperConnection Watcher:127.0.0.1:52162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.833; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.835; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.839; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.841; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.843; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@15ea3fe name:ZooKeeperConnection Watcher:127.0.0.1:52162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.843; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.845; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.851; org.apache.solr.handler.admin.CollectionsHandler; Creating Collection : replicationFactor=2&wt=javabin&maxShardsPerNode=1&version=2&action=CREATE&numShards=1&name=awholynewcollection_0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.852; org.apache.solr.cloud.DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.853; org.apache.solr.cloud.OverseerCollectionProcessor; Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000002 message:{
[junit4:junit4] 1> "replicationFactor":"2",
[junit4:junit4] 1> "maxShardsPerNode":"1",
[junit4:junit4] 1> "operation":"createcollection",
[junit4:junit4] 1> "createNodeSet":null,
[junit4:junit4] 1> "numShards":"1",
[junit4:junit4] 1> "name":"awholynewcollection_0"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.853; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_0_shard1_replica1 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:32786_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.853; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_0_shard1_replica2 as part of slice shard1 of collection awholynewcollection_0 on 127.0.0.1:37645_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.854; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_0_shard1_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_0_shard1_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.855; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.855; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_0_shard1_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_0_shard1_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.855; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.855; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.856; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.856; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:awholynewcollection_0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.857; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.857; org.apache.solr.cloud.ZkController; Only one config set found in zk - using it:conf1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.858; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/awholynewcollection_0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.859; org.apache.solr.cloud.ZkController; Only one config set found in zk - using it:conf1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.859; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/awholynewcollection_0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.860; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.861; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_0_shard1_replica2/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.863; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.864; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_0_shard1_replica1/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.889; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.898; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.920; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.922; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.926; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.934; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.935; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:50.940; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.285; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.293; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.296; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.308; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.312; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.316; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 18:27:51.317; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 18:27:51.318; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.318; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 18:27:51.319; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 18:27:51.320; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.320; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.321; org.apache.solr.core.SolrCore; [awholynewcollection_0_shard1_replica2] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_0_shard1_replica2/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_0_shard1_replica2/data/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.321; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13918b0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.322; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.323; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_0_shard1_replica2/data
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.323; org.apache.solr.core.SolrCore; New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_0_shard1_replica2/data/index/
[junit4:junit4] 1> WARN - 2013-04-14 18:27:51.324; org.apache.solr.core.SolrCore; [awholynewcollection_0_shard1_replica2] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_0_shard1_replica2/data/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.324; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_0_shard1_replica2/data/index
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.326; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@442d88 lockFactory=org.apache.lucene.store.NativeFSLockFactory@17f6ffd),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.327; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.329; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.330; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.331; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.331; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.332; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.333; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.333; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.334; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.334; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.338; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.341; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@692739 main
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.341; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_0_shard1_replica2/data/tlog
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.342; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.342; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.346; org.apache.solr.core.SolrCore; [awholynewcollection_0_shard1_replica2] Registered new searcher Searcher@692739 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.349; org.apache.solr.cloud.ZkController; publishing core=awholynewcollection_0_shard1_replica2 state=down
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.350; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.356; org.apache.solr.schema.IndexSchema; default search field in schema is text
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.358; org.apache.solr.schema.IndexSchema; unique key field: id
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.368; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.372; org.apache.solr.schema.FileExchangeRateProvider; Reloading exchange rates from file currency.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.375; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 18:27:51.376; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 18:27:51.376; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.377; org.apache.solr.schema.OpenExchangeRatesOrgProvider; Reloading exchange rates from open-exchange-rates.json
[junit4:junit4] 1> WARN - 2013-04-14 18:27:51.378; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Unknown key IMPORTANT NOTE
[junit4:junit4] 1> WARN - 2013-04-14 18:27:51.378; org.apache.solr.schema.OpenExchangeRatesOrgProvider$OpenExchangeRates; Expected key, got STRING
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.378; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.379; org.apache.solr.core.SolrCore; [awholynewcollection_0_shard1_replica1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_0_shard1_replica1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_0_shard1_replica1/data/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.379; org.apache.solr.core.JmxMonitoredMap; JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13918b0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.379; org.apache.solr.core.SolrCore; org.apache.solr.core.MockDirectoryFactory
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.380; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_0_shard1_replica1/data
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.380; org.apache.solr.core.SolrCore; New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_0_shard1_replica1/data/index/
[junit4:junit4] 1> WARN - 2013-04-14 18:27:51.381; org.apache.solr.core.SolrCore; [awholynewcollection_0_shard1_replica1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_0_shard1_replica1/data/index' doesn't exist. Creating new index...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.381; org.apache.solr.core.CachingDirectoryFactory; return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_0_shard1_replica1/data/index
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.383; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1530c42 lockFactory=org.apache.lucene.store.NativeFSLockFactory@aa919c),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.383; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.385; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.385; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.386; org.apache.solr.update.processor.UpdateRequestProcessorChain; inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.386; org.apache.solr.core.RequestHandlers; created standard: solr.StandardRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.387; org.apache.solr.core.RequestHandlers; created /get: solr.RealTimeGetHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.387; org.apache.solr.core.RequestHandlers; adding lazy requestHandler: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.387; org.apache.solr.core.RequestHandlers; created /replication: solr.ReplicationHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.388; org.apache.solr.core.RequestHandlers; created /update: solr.UpdateRequestHandler
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.388; org.apache.solr.core.RequestHandlers; created /admin/: org.apache.solr.handler.admin.AdminHandlers
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.391; org.apache.solr.handler.loader.XMLLoader; xsltCacheLifetimeSeconds=60
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.393; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@1aba1c9 main
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.394; org.apache.solr.update.UpdateHandler; Clearing tlog files, tlogDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_0_shard1_replica1/data/tlog
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.394; org.apache.solr.update.CommitTracker; Hard AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.394; org.apache.solr.update.CommitTracker; Soft AutoCommit: disabled
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.397; org.apache.solr.core.SolrCore; [awholynewcollection_0_shard1_replica1] Registered new searcher Searcher@1aba1c9 main{StandardDirectoryReader(segments_1:1)}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:51.399; org.apache.solr.cloud.ZkController; publishing core=awholynewcollection_0_shard1_replica1 state=down
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.232; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.233; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "state":"recovering",
[junit4:junit4] 1> "collection":"nodes_used_collection",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:44653__nodes_used_collection_shard1_replica2",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:44653_",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "core":"nodes_used_collection_shard1_replica2",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:44653"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.236; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "collection":"nodes_used_collection",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:51793__nodes_used_collection_shard1_replica1",
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:51793_",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "core":"nodes_used_collection_shard1_replica1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:51793"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.238; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "state":"active",
[junit4:junit4] 1> "collection":"nodes_used_collection",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:32786__nodes_used_collection_shard2_replica2",
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:32786_",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "core":"nodes_used_collection_shard2_replica2",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:32786"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.240; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=2 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "state":"recovering",
[junit4:junit4] 1> "collection":"nodes_used_collection",
[junit4:junit4] 1> "core_node_name":"127.0.0.1:46976__nodes_used_collection_shard2_replica1",
[junit4:junit4] 1> "shard":"shard2",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:46976_",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "numShards":"2",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "core":"nodes_used_collection_shard2_replica1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:46976"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.242; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=1 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "collection":"awholynewcollection_0",
[junit4:junit4] 1> "core_node_name":null,
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:37645_",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "numShards":"1",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "core":"awholynewcollection_0_shard1_replica2",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:37645"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.243; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Create collection awholynewcollection_0 with numShards 1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.245; org.apache.solr.cloud.Overseer$ClusterStateUpdater; Update state numShards=1 message={
[junit4:junit4] 1> "roles":null,
[junit4:junit4] 1> "state":"down",
[junit4:junit4] 1> "collection":"awholynewcollection_0",
[junit4:junit4] 1> "core_node_name":null,
[junit4:junit4] 1> "shard":"shard1",
[junit4:junit4] 1> "shard_range":null,
[junit4:junit4] 1> "node_name":"127.0.0.1:32786_",
[junit4:junit4] 1> "operation":"state",
[junit4:junit4] 1> "numShards":"1",
[junit4:junit4] 1> "shard_state":"active",
[junit4:junit4] 1> "core":"awholynewcollection_0_shard1_replica1",
[junit4:junit4] 1> "base_url":"http://127.0.0.1:32786"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.247; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.248; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.249; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.249; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.249; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.249; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.249; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.248; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.350; org.apache.solr.core.CoreContainer; registering core: awholynewcollection_0_shard1_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.351; org.apache.solr.cloud.ZkController; Register replica - core:awholynewcollection_0_shard1_replica2 address:http://127.0.0.1:37645 collection:awholynewcollection_0 shard:shard1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.351; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/awholynewcollection_0/leader_elect/shard1/election
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.357; org.apache.solr.cloud.ShardLeaderElectionContext; Running the leader process.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.359; org.apache.solr.cloud.ShardLeaderElectionContext; Waiting until we see more replicas up: total=2 found=1 timeoutin=179999
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.401; org.apache.solr.core.CoreContainer; registering core: awholynewcollection_0_shard1_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.401; org.apache.solr.cloud.ZkController; Register replica - core:awholynewcollection_0_shard1_replica1 address:http://127.0.0.1:32786 collection:awholynewcollection_0 shard:shard1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.768; org.apache.solr.handler.admin.CoreAdminHandler; Waited coreNodeName: 127.0.0.1:44653__nodes_used_collection_shard1_replica2, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.769; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={checkLive=true&state=recovering&coreNodeName=127.0.0.1:44653__nodes_used_collection_shard1_replica2&wt=javabin&onlyIfLeader=true&version=2&nodeName=127.0.0.1:44653_&action=PREPRECOVERY&core=nodes_used_collection_shard1_replica1} status=0 QTime=2001
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.782; org.apache.solr.handler.admin.CoreAdminHandler; Waited coreNodeName: 127.0.0.1:46976__nodes_used_collection_shard2_replica1, state: recovering, checkLive: true, onlyIfLeader: true for: 2 seconds.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.782; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={checkLive=true&state=recovering&coreNodeName=127.0.0.1:46976__nodes_used_collection_shard2_replica1&wt=javabin&onlyIfLeader=true&version=2&nodeName=127.0.0.1:46976_&action=PREPRECOVERY&core=nodes_used_collection_shard2_replica2} status=0 QTime=2001
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.860; org.apache.solr.cloud.ShardLeaderElectionContext; Enough replicas found to continue.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.861; org.apache.solr.cloud.ShardLeaderElectionContext; I may be the new leader - try and sync
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.861; org.apache.solr.cloud.SyncStrategy; Sync replicas to http://127.0.0.1:37645/awholynewcollection_0_shard1_replica2/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.862; org.apache.solr.update.PeerSync; PeerSync: core=awholynewcollection_0_shard1_replica2 url=http://127.0.0.1:37645 START replicas=[http://127.0.0.1:32786/awholynewcollection_0_shard1_replica1/] nUpdates=100
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.863; org.apache.solr.update.PeerSync; PeerSync: core=awholynewcollection_0_shard1_replica2 url=http://127.0.0.1:37645 DONE. We have no versions. sync failed.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.863; org.apache.solr.cloud.SyncStrategy; Leader's attempt to sync with shard failed, moving to the next canidate
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.864; org.apache.solr.cloud.ShardLeaderElectionContext; We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.864; org.apache.solr.cloud.ShardLeaderElectionContext; I am the new leader: http://127.0.0.1:37645/awholynewcollection_0_shard1_replica2/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.864; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/awholynewcollection_0/leaders/shard1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:52.868; org.apache.solr.core.SolrCore; [awholynewcollection_0_shard1_replica1] webapp= path=/get params={wt=javabin&version=2&getVersions=100&distrib=false&qt=/get} status=0 QTime=1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.764; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.771; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.771; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.771; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.771; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.771; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.771; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.771; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.771; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.805; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:32786/awholynewcollection_0_shard1_replica1/ and leader is http://127.0.0.1:37645/awholynewcollection_0_shard1_replica2/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.805; org.apache.solr.cloud.ZkController; No LogReplay needed for core=awholynewcollection_0_shard1_replica1 baseURL=http://127.0.0.1:32786
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.806; org.apache.solr.cloud.ZkController; Core needs to recover:awholynewcollection_0_shard1_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.806; org.apache.solr.update.DefaultSolrCoreState; Running recovery - first canceling any ongoing recovery
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.806; org.apache.solr.cloud.RecoveryStrategy; Starting recovery process. core=awholynewcollection_0_shard1_replica1 recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.807; org.apache.solr.cloud.RecoveryStrategy; ###### startupVersions=[]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.808; org.apache.solr.cloud.ZkController; publishing core=awholynewcollection_0_shard1_replica1 state=recovering
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.808; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.810; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.811; org.apache.solr.core.CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.816; org.apache.solr.handler.admin.CoreAdminHandler; Going to wait for coreNodeName: 127.0.0.1:32786__awholynewcollection_0_shard1_replica1, state: recovering, checkLive: true, onlyIfLeader: true
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.821; org.apache.solr.core.SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.822; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={collection=awholynewcollection_0&shard=shard1&wt=javabin&version=2&action=CREATE&numShards=1&qt=/admin/cores&name=awholynewcollection_0_shard1_replica1} status=0 QTime=2967
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.823; org.apache.solr.cloud.ZkController; We are http://127.0.0.1:37645/awholynewcollection_0_shard1_replica2/ and leader is http://127.0.0.1:37645/awholynewcollection_0_shard1_replica2/
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.823; org.apache.solr.cloud.ZkController; No LogReplay needed for core=awholynewcollection_0_shard1_replica2 baseURL=http://127.0.0.1:37645
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.824; org.apache.solr.cloud.ZkController; I am the leader, no recovery necessary
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.824; org.apache.solr.cloud.ZkController; publishing core=awholynewcollection_0_shard1_replica2 state=active
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.826; org.apache.solr.common.cloud.ZkStateReader; Updating cloud state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.827; org.apache.solr.core.CoreContainer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.837; org.apache.solr.core.SolrXMLSerializer; Persisting cores config to /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/solr.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.839; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/cores params={collection=awholynewcollection_0&shard=shard1&wt=javabin&version=2&action=CREATE&numShards=1&qt=/admin/cores&name=awholynewcollection_0_shard1_replica2} status=0 QTime=2985
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.840; org.apache.solr.cloud.OverseerCollectionProcessor; Finished create command on all shards for collection: awholynewcollection_0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.857; org.apache.solr.cloud.DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/collection-queue-work/qnr-0000000002 state: SyncConnected type NodeDataChanged
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.858; org.apache.solr.cloud.DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.858; org.apache.solr.cloud.OverseerCollectionProcessor; Overseer Collection Processor: Message id:/overseer/collection-queue-work/qn-0000000002 complete, response:{success={null={responseHeader={status=0,QTime=2967},core=awholynewcollection_0_shard1_replica1,saved=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/solr.xml},null={responseHeader={status=0,QTime=2985},core=awholynewcollection_0_shard1_replica2,saved=/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/solr.xml}}}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.860; org.apache.solr.servlet.SolrDispatchFilter; [admin] webapp=null path=/admin/collections params={replicationFactor=2&wt=javabin&maxShardsPerNode=1&version=2&action=CREATE&numShards=1&name=awholynewcollection_0} status=0 QTime=3009
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.861; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.863; org.apache.solr.common.cloud.ConnectionManager; Waiting for client to connect to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.864; org.apache.solr.common.cloud.ConnectionManager; Watcher org.apache.solr.common.cloud.ConnectionManager@c40e8c name:ZooKeeperConnection Watcher:127.0.0.1:52162/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.864; org.apache.solr.common.cloud.ConnectionManager; Client is connected to ZooKeeper
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.865; org.apache.solr.common.cloud.ZkStateReader; Updating cluster state from ZooKeeper...
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.869; org.apache.solr.handler.admin.CollectionsHandler; Creating Collection : replicationFactor=4&wt=javabin&maxShardsPerNode=4&version=2&action=CREATE&numShards=4&name=awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.870; org.apache.solr.cloud.DistributedQueue$LatchChildWatcher; Watcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.871; org.apache.solr.cloud.OverseerCollectionProcessor; Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000004 message:{
[junit4:junit4] 1> "replicationFactor":"4",
[junit4:junit4] 1> "maxShardsPerNode":"4",
[junit4:junit4] 1> "operation":"createcollection",
[junit4:junit4] 1> "createNodeSet":null,
[junit4:junit4] 1> "numShards":"4",
[junit4:junit4] 1> "name":"awholynewcollection_1"}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.871; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard1_replica1 as part of slice shard1 of collection awholynewcollection_1 on 127.0.0.1:44653_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.871; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard1_replica2 as part of slice shard1 of collection awholynewcollection_1 on 127.0.0.1:37645_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.872; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard1_replica3 as part of slice shard1 of collection awholynewcollection_1 on 127.0.0.1:46976_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.872; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard1_replica4 as part of slice shard1 of collection awholynewcollection_1 on 127.0.0.1:32786_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.872; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard2_replica1 as part of slice shard2 of collection awholynewcollection_1 on 127.0.0.1:51793_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.873; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard2_replica2 as part of slice shard2 of collection awholynewcollection_1 on 127.0.0.1:44653_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.873; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard2_replica3 as part of slice shard2 of collection awholynewcollection_1 on 127.0.0.1:37645_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.873; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard1_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/awholynewcollection_1_shard1_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.873; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.873; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard1_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_1_shard1_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.873; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard2_replica4 as part of slice shard2 of collection awholynewcollection_1 on 127.0.0.1:46976_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.874; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.874; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.874; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard2_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/awholynewcollection_1_shard2_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.874; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard2_replica3' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_1_shard2_replica3
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.874; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard1_replica4' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_1_shard1_replica4
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.876; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.874; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard1_replica3' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/awholynewcollection_1_shard1_replica3
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.877; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.874; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard2_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/awholynewcollection_1_shard2_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.877; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.877; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.876; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.875; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.875; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.879; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.875; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.875; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard2_replica4' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/awholynewcollection_1_shard2_replica4
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.880; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.874; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard3_replica1 as part of slice shard3 of collection awholynewcollection_1 on 127.0.0.1:32786_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.880; org.apache.solr.cloud.ZkController; Only one config set found in zk - using it:conf1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.879; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.879; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.878; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.878; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.878; org.apache.solr.cloud.ZkController; Only one config set found in zk - using it:conf1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.877; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.883; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.883; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.882; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.882; org.apache.solr.cloud.ZkController; Only one config set found in zk - using it:conf1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.882; org.apache.solr.cloud.ZkController; Only one config set found in zk - using it:conf1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.882; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.885; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.881; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard3_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_1_shard3_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.881; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.886; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_1_shard1_replica4/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.881; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard3_replica2 as part of slice shard3 of collection awholynewcollection_1 on 127.0.0.1:51793_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.881; org.apache.solr.cloud.ZkController; Creating collection in ZooKeeper:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.886; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard3_replica3 as part of slice shard3 of collection awholynewcollection_1 on 127.0.0.1:44653_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.885; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.887; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard3_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/awholynewcollection_1_shard3_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.888; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.885; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.884; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.884; org.apache.solr.cloud.ZkController; Only one config set found in zk - using it:conf1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.884; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.883; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.889; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.890; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.890; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.889; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.891; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.888; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.892; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.888; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard3_replica3' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/awholynewcollection_1_shard3_replica3
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.892; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.893; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/awholynewcollection_1_shard3_replica2/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.887; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard3_replica4 as part of slice shard3 of collection awholynewcollection_1 on 127.0.0.1:37645_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.887; org.apache.solr.cloud.ZkController; Looking for collection configName
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.887; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.894; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard4_replica1 as part of slice shard4 of collection awholynewcollection_1 on 127.0.0.1:46976_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.894; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/awholynewcollection_1_shard2_replica2/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.895; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/awholynewcollection_1_shard1_replica1/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.896; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard4_replica1' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/awholynewcollection_1_shard4_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.896; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.897; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.898; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.899; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/awholynewcollection_1_shard4_replica1/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.892; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.892; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_1_shard3_replica1/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.902; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.902; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.891; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/awholynewcollection_1_shard2_replica1/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.891; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.891; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.891; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.895; org.apache.solr.common.cloud.SolrZkClient; makePath: /collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.895; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard3_replica4' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_1_shard3_replica4
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.895; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard4_replica2 as part of slice shard4 of collection awholynewcollection_1 on 127.0.0.1:32786_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.919; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_1_shard1_replica2/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.920; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_1_shard2_replica3/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.919; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.918; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/awholynewcollection_1_shard3_replica3/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.921; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard4_replica3 as part of slice shard4 of collection awholynewcollection_1 on 127.0.0.1:51793_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.919; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/awholynewcollection_1_shard1_replica3/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.922; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard4_replica2' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_1_shard4_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.923; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.922; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.925; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.923; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.923; org.apache.solr.cloud.OverseerCollectionProcessor; Creating shard awholynewcollection_1_shard4_replica4 as part of slice shard4 of collection awholynewcollection_1 on 127.0.0.1:44653_
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.927; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.927; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty2-1365964064326/awholynewcollection_1_shard2_replica4/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.928; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.929; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard4_replica3' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/awholynewcollection_1_shard4_replica3
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.929; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.929; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-controljetty-1365964063003/awholynewcollection_1_shard4_replica2/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.931; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty3-1365964065468/awholynewcollection_1_shard3_replica4/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.934; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.935; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.935; org.apache.solr.core.CoreContainer; Creating SolrCore 'awholynewcollection_1_shard4_replica4' using instanceDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/awholynewcollection_1_shard4_replica4
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.936; org.apache.solr.cloud.ZkController; Check for collection zkNode:awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.937; org.apache.solr.cloud.ZkController; Collection zkNode exists
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.938; org.apache.solr.cloud.ZkController; Load collection config from:/collections/awholynewcollection_1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.937; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/awholynewcollection_1_shard4_replica3/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.939; org.apache.solr.core.SolrResourceLoader; new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty1-1365964063196/awholynewcollection_1_shard4_replica4/'
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.971; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.973; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.982; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.983; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.993; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.996; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:53.997; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.009; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.043; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.059; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.060; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.060; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.069; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.091; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.092; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.094; org.apache.solr.core.SolrConfig; Using Lucene MatchVersion: LUCENE_50
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.125; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.127; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.134; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.143; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.173; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.181; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.182; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.194; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.195; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.199; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.201; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.194; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.210; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.211; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.212; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.214; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.215; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.220; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.221; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.221; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.227; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.238; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.241; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.239; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.246; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.248; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.248; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.255; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.258; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.262; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.266; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.267; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.291; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.291; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.296; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.299; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.300; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.302; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.323; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.397; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.399; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.406; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.419; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.420; org.apache.solr.core.SolrConfig; Loaded SolrConfig: solrconfig.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.425; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.431; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.443; org.apache.solr.schema.IndexSchema; Reading Solr Schema from schema.xml
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.471; org.apache.solr.schema.IndexSchema; Schema name=test
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.769; org.apache.solr.cloud.RecoveryStrategy; Attempting to PeerSync from http://127.0.0.1:51793/nodes_used_collection_shard1_replica1/ core=nodes_used_collection_shard1_replica2 - recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.770; org.apache.solr.update.PeerSync; PeerSync: core=nodes_used_collection_shard1_replica2 url=http://127.0.0.1:44653 START replicas=[http://127.0.0.1:51793/nodes_used_collection_shard1_replica1/] nUpdates=100
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.791; org.apache.solr.cloud.RecoveryStrategy; Attempting to PeerSync from http://127.0.0.1:32786/nodes_used_collection_shard2_replica2/ core=nodes_used_collection_shard2_replica1 - recoveringAfterStartup=true
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.792; org.apache.solr.update.PeerSync; PeerSync: core=nodes_used_collection_shard2_replica1 url=http://127.0.0.1:46976 START replicas=[http://127.0.0.1:32786/nodes_used_collection_shard2_replica2/] nUpdates=100
[junit4:junit4] 1> WARN - 2013-04-14 18:27:54.792; org.apache.solr.update.PeerSync; no frame of reference to tell of we've missed updates
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.793; org.apache.solr.cloud.RecoveryStrategy; PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.793; org.apache.solr.cloud.RecoveryStrategy; Starting Replication Recovery. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.793; org.apache.solr.cloud.RecoveryStrategy; Begin buffering updates. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.793; org.apache.solr.update.UpdateLog; Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 1> WARN - 2013-04-14 18:27:54.793; org.apache.solr.update.PeerSync; no frame of reference to tell of we've missed updates
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.794; org.apache.solr.cloud.RecoveryStrategy; PeerSync Recovery was not successful - trying replication. core=nodes_used_collection_shard1_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.795; org.apache.solr.cloud.RecoveryStrategy; Starting Replication Recovery. core=nodes_used_collection_shard1_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.795; org.apache.solr.cloud.RecoveryStrategy; Begin buffering updates. core=nodes_used_collection_shard1_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.796; org.apache.solr.update.UpdateLog; Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.796; org.apache.solr.cloud.RecoveryStrategy; Attempting to replicate from http://127.0.0.1:51793/nodes_used_collection_shard1_replica1/. core=nodes_used_collection_shard1_replica2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.796; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.793; org.apache.solr.cloud.RecoveryStrategy; Attempting to replicate from http://127.0.0.1:32786/nodes_used_collection_shard2_replica2/. core=nodes_used_collection_shard2_replica1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.797; org.apache.solr.client.solrj.impl.HttpClientUtil; Creating new http client, config:maxConnections=128&maxConnectionsPerHost=32&followRedirects=false
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.804; org.apache.solr.core.SolrCore; [nodes_used_collection_shard2_replica2] webapp= path=/get params={wt=javabin&version=2&getVersions=100&distrib=false&qt=/get} status=0 QTime=0
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.806; org.apache.solr.update.DirectUpdateHandler2; start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.811; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onInit: commits:num=1
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fcd4f1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18744cc),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.812; org.apache.solr.core.SolrDeletionPolicy; newest commit = 1[segments_1]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.813; org.apache.solr.core.SolrDeletionPolicy; SolrDeletionPolicy.onCommit: commits:num=2
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fcd4f1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18744cc),segFN=segments_1,generation=1,filenames=[segments_1]
[junit4:junit4] 1> commit{dir=MockDirWrapper(org.apache.lucene.store.RAMDirectory@1fcd4f1 lockFactory=org.apache.lucene.store.NativeFSLockFactory@18744cc),segFN=segments_2,generation=2,filenames=[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.813; org.apache.solr.core.SolrDeletionPolicy; newest commit = 2[segments_2]
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.814; org.apache.solr.search.SolrIndexSearcher; Opening Searcher@8e8ecc realtime
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.814; org.apache.solr.update.DirectUpdateHandler2; end_commit_flush
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.815; org.apache.solr.update.processor.LogUpdateProcessor; [nodes_used_collection_shard2_replica2] webapp= path=/update params={commit_end_point=true&commit=true&waitSearcher=true&wt=javabin&version=2&openSearcher=false&softCommit=false} {commit=} 0 9
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.815; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for 10000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.816; org.apache.solr.handler.SnapPuller; No value set for 'pollInterval'. Timer Task not started.
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.821; org.apache.solr.handler.ReplicationHandler; Commits will be reserved for 10000
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.821; org.apache.solr.core.SolrCore; [nodes_used_collection_shard2_replica2] webapp= path=/replication params={wt=javabin&version=2&qt=/replication&command=indexversion} status=0 QTime=4
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.822; org.apache.solr.handler.SnapPuller; Master's generation: 2
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.822; org.apache.solr.handler.SnapPuller; Slave's generation: 1
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.823; org.apache.solr.handler.SnapPuller; Starting replication process
[junit4:junit4] 1> INFO - 2013-04-14 18:27:54.828; org.apache.solr.core.SolrCore; [nodes_used_collection_shard2_replica2] webapp= path=/replication params={wt=javabin&version=2&generation=2&qt=/replication&command=filelist} status=0 QTime=4
[...truncated too long message...]
cene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/awholynewcollection_2_shard1_replica2/data/index;done=false>>]
[junit4:junit4] 1> INFO - 2013-04-14 18:32:16.992; org.apache.solr.core.CachingDirectoryFactory; Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./org.apache.solr.cloud.CollectionsAPIDistributedZkTest-jetty4-1365964066612/awholynewcollection_2_shard1_replica2/data/index
[junit4:junit4] 1> INFO - 2013-04-14 18:32:16.994; org.apache.solr.cloud.Overseer$ClusterStateUpdater; According to ZK I (id=89519820811272204-127.0.0.1:51793_-n_0000000004) am no longer a leader.
[junit4:junit4] 1> INFO - 2013-04-14 18:32:16.995; org.apache.solr.common.cloud.ZkStateReader$2; A cluster state change: WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes, has occurred - updating... (live nodes size: 1)
[junit4:junit4] 1> INFO - 2013-04-14 18:32:17.016; org.eclipse.jetty.server.handler.ContextHandler; stopped o.e.j.s.ServletContextHandler{/,null}
[junit4:junit4] 1> INFO - 2013-04-14 18:32:17.071; org.apache.solr.SolrTestCaseJ4; ###Ending testDistribSearch
[junit4:junit4] 1> INFO - 2013-04-14 18:32:17.073; org.apache.solr.cloud.ZkTestServer; connecting to 127.0.0.1:52162 52162
[junit4:junit4] 2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build/solr-core/test/J1/./solrtest-CollectionsAPIDistributedZkTest-1365964062670
[junit4:junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=DDBBB8F1126DC87F -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar_JO -Dtests.timezone=Africa/Lome -Dtests.file.encoding=UTF-8
[junit4:junit4] FAILURE 274s J1 | CollectionsAPIDistributedZkTest.testDistribSearch <<<
[junit4:junit4] > Throwable #1: java.lang.AssertionError: There are still nodes recoverying - waited for 230 seconds
[junit4:junit4] > at __randomizedtesting.SeedInfo.seed([DDBBB8F1126DC87F:5C5D36E96532A843]:0)
[junit4:junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:173)
[junit4:junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:131)
[junit4:junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:126)
[junit4:junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:483)
[junit4:junit4] > at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:146)
[junit4:junit4] > at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:806)
[junit4:junit4] > at java.lang.Thread.run(Thread.java:722)
[junit4:junit4] 1> INFO - 2013-04-14 18:32:17.116; org.apache.solr.SolrTestCaseJ4; ###deleteCore
[junit4:junit4] 2> 274454 T2256 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 2 leaked thread(s).
[junit4:junit4] 1> WARN - 2013-04-14 18:32:18.295; org.apache.solr.common.cloud.ZkStateReader$2; ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 1> WARN - 2013-04-14 18:32:18.295; org.apache.solr.common.cloud.ZkStateReader$3; ZooKeeper watch triggered, but Solr cannot talk to ZK
[junit4:junit4] 1> WARN - 2013-04-14 18:32:18.295; org.apache.solr.cloud.LeaderElector$1;
[junit4:junit4] 1> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewcollection_2/leader_elect/shard1/election
[junit4:junit4] 1> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 1> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 1> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 1> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 1> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 1> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 1> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 1> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 1> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 1> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 1> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 1> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 1> WARN - 2013-04-14 18:32:18.296; org.apache.solr.cloud.LeaderElector$1;
[junit4:junit4] 1> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/awholynewcollection_2/leader_elect/shard2/election
[junit4:junit4] 1> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4:junit4] 1> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4:junit4] 1> at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
[junit4:junit4] 1> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:235)
[junit4:junit4] 1> at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:232)
[junit4:junit4] 1> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:65)
[junit4:junit4] 1> at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:232)
[junit4:junit4] 1> at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:84)
[junit4:junit4] 1> at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
[junit4:junit4] 1> at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:129)
[junit4:junit4] 1> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
[junit4:junit4] 1> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
[junit4:junit4] 2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=381), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=381)), sim=DefaultSimilarity, locale=ar_JO, timezone=Africa/Lome
[junit4:junit4] 2> NOTE: Linux 3.2.0-40-generic i386/Oracle Corporation 1.8.0-ea (32-bit)/cpus=8,threads=1,free=260051272,total=499122176
[junit4:junit4] 2> NOTE: All tests run in this JVM: [TestArbitraryIndexDir, ConvertedLegacyTest, TestManagedSchema, ClusterStateTest, TermVectorComponentDistributedTest, SpellPossibilityIteratorTest, TestStressReorder, TestPostingsSolrHighlighter, TestSolrJ, TestSchemaNameResource, TestShardHandlerFactory, TestCollationField, TestDistributedSearch, TestPartialUpdateDeduplication, TimeZoneUtilsTest, SpellCheckComponentTest, NumericFieldsTest, FileUtilsTest, XsltUpdateRequestHandlerTest, TestHashPartitioner, TestUtils, ZkSolrClientTest, CoreContainerCoreInitFailuresTest, TestSolrQueryParserResource, TestCodecSupport, TestFieldResource, OpenExchangeRatesOrgProviderTest, NoCacheHeaderTest, UniqFieldsUpdateProcessorFactoryTest, TestAnalyzedSuggestions, TestClassNameShortening, SolrIndexConfigTest, DefaultValueUpdateProcessorTest, TestWordDelimiterFilterFactory, SolrCoreCheckLockOnStartupTest, BadIndexSchemaTest, SuggesterTSTTest, TestQuerySenderNoQuery, TestBinaryField, FullSolrCloudDistribCmdsTest, CurrencyFieldOpenExchangeTest, HighlighterTest, BasicZkTest, ZkNodePropsTest, DisMaxRequestHandlerTest, ShardRoutingCustomTest, LeaderElectionIntegrationTest, ChaosMonkeySafeLeaderTest, PathHierarchyTokenizerFactoryTest, TestLuceneMatchVersion, SpellCheckCollatorTest, TestStressLucene, TestCSVLoader, XmlUpdateRequestHandlerTest, TestElisionMultitermQuery, TestJmxIntegration, URLClassifyProcessorTest, DocValuesMultiTest, MultiTermTest, TestNumberUtils, TestLazyCores, TestJmxMonitoredMap, TestRangeQuery, TestLMJelinekMercerSimilarityFactory, SignatureUpdateProcessorFactoryTest, OpenCloseCoreStressTest, QueryResultKeyTest, TestBM25SimilarityFactory, IndexSchemaRuntimeFieldTest, SolrRequestParserTest, TestCopyFieldCollectionResource, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
[junit4:junit4] Completed on J1 in 275.74s, 1 test, 1 failure <<< FAILURES!
[...truncated 455 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:375: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:355: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/build.xml:183: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/solr/common-build.xml:438: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1231: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:875: There were test failures: 289 suites, 1201 tests, 1 failure, 13 ignored (7 assumptions)
Total time: 41 minutes 46 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Description set: Java: 32bit/jdk1.8.0-ea-b84 -server -XX:+UseG1GC
Email was triggered for: Failure
Sending email for trigger: Failure