You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/03/31 19:35:51 UTC

[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0) - Build # 9849 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9849/
Java: 32bit/jdk1.8.0 -server -XX:+UseParallelGC

1 tests failed.
REGRESSION:  org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([716E708A95530CDC:F088FE92E20C6CE0]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertTrue(Assert.java:54)
	at org.apache.solr.client.solrj.impl.CloudSolrServerTest.doTest(CloudSolrServerTest.java:256)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:826)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:862)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 12206 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
   [junit4]   2> log4j:WARN No such property [conversionPattern] in org.apache.solr.util.SolrLogLayout.
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1396287262795
   [junit4]   2> 1170 T12 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1201 T12 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 1208 T12 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
   [junit4]   2> 1253 T12 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 1257 T12 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 1261 T13 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1363 T12 oasc.ZkTestServer.run start zk server on port:41112
   [junit4]   2> 1404 T12 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1469 T19 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57ce27 name:ZooKeeperConnection Watcher:127.0.0.1:41112 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1470 T12 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1472 T12 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1517 T12 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1519 T21 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1399b79 name:ZooKeeperConnection Watcher:127.0.0.1:41112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1519 T12 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1529 T12 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 1537 T12 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 1544 T12 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 1548 T12 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 1553 T12 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1575 T12 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1582 T12 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1583 T12 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 1587 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
   [junit4]   2> 1587 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
   [junit4]   2> 1588 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
   [junit4]   2> 1589 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
   [junit4]   2> 1589 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
   [junit4]   2> 1590 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
   [junit4]   2> 1591 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
   [junit4]   2> 1592 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
   [junit4]   2> 1592 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
   [junit4]   2> 1976 T12 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2118 T12 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:36133
   [junit4]   2> 2188 T12 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2212 T12 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2213 T12 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1396287263530
   [junit4]   2> 2213 T12 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1396287263530/'
   [junit4]   2> 2349 T12 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1396287263530/solr.xml
   [junit4]   2> 2510 T12 oasc.CoreContainer.<init> New CoreContainer 28096326
   [junit4]   2> 2511 T12 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1396287263530/]
   [junit4]   2> 2543 T12 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2544 T12 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2550 T12 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2551 T12 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2551 T12 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2552 T12 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2552 T12 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2553 T12 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2553 T12 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2713 T12 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2715 T12 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2717 T12 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2718 T12 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41112/solr
   [junit4]   2> 2728 T12 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2729 T12 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2736 T33 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16f1717 name:ZooKeeperConnection Watcher:127.0.0.1:41112 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2736 T12 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2745 T12 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2750 T35 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a3cb91 name:ZooKeeperConnection Watcher:127.0.0.1:41112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2751 T12 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2766 T12 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 2788 T12 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 2796 T12 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 2801 T12 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 2806 T12 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 2818 T12 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 2821 T12 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36133_
   [junit4]   2> 2825 T12 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:36133_
   [junit4]   2> 2833 T12 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 2837 T12 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 2864 T12 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:36133_
   [junit4]   2> 2865 T12 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2883 T12 oasc.Overseer.start Overseer (id=91507082084548611-127.0.0.1:36133_-n_0000000000) starting
   [junit4]   2> 2909 T12 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 2966 T37 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2967 T12 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 2978 T12 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 2979 T37 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 2982 T12 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3034 T36 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 3131 T38 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 3132 T38 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 3136 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3136 T38 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 3139 T36 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 3140 T36 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:36133",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:36133_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 3141 T36 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 3148 T36 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 3151 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3155 T35 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 4138 T38 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 4138 T38 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 4139 T38 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 4139 T38 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 4141 T38 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 4141 T38 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1396287263530/collection1/'
   [junit4]   2> 4197 T38 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 4267 T38 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4280 T38 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 4364 T38 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 4386 T38 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.IntField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 4397 T38 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.LongField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 4405 T38 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.FloatField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 4414 T38 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.DoubleField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 4463 T38 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableIntField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 4471 T38 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableLongField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 4480 T38 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableFloatField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 4489 T38 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.SortableDoubleField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 4496 T38 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.BCDIntField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 4503 T38 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.BCDLongField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 4510 T38 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.BCDStrField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 4576 T38 oasc.SolrResourceLoader.findClass WARN Solr loaded a deprecated plugin/analysis class [solr.DateField]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 4798 T38 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 4801 T38 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 4807 T38 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 4807 T38 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 4826 T38 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 4860 T38 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 4874 T38 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1396287263530/collection1/, dataDir=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/control/data/
   [junit4]   2> 4876 T38 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 4916 T38 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/control/data
   [junit4]   2> 4917 T38 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/control/data/index/
   [junit4]   2> 4918 T38 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 4936 T38 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/control/data/index
   [junit4]   2> 5006 T38 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@89a378 lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/control/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 5008 T38 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 5268 T38 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 5290 T38 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 5291 T38 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 5292 T38 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 5296 T38 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 5317 T38 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 5319 T38 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 5326 T38 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 5369 T38 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 5424 T38 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 5425 T38 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 5430 T38 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@89a378 lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/control/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 5431 T38 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 5553 T38 oass.SolrIndexSearcher.<init> Opening Searcher@12551b4[collection1] main
   [junit4]   2> 5620 T38 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 5620 T39 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@12551b4[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 5630 T42 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:36133 collection:control_collection shard:shard1
   [junit4]   2> 5630 T12 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0
   [junit4]   2> 5631 T12 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 5666 T42 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 5691 T44 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c928b4 name:ZooKeeperConnection Watcher:127.0.0.1:41112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 5691 T12 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 5692 T12 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 5700 T12 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 5704 T42 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 5707 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 5712 T42 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 5712 T36 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 5713 T42 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 5716 T12 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 5718 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 5720 T35 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 5720 T44 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> ASYNC  NEW_CORE C0 name=collection1 org.apache.solr.core.SolrCore@1fe6678 url=http://127.0.0.1:36133/collection1 node=127.0.0.1:36133_ C0_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:36133, core=collection1, node_name=127.0.0.1:36133_}
   [junit4]   2> 5730 T42 C0 P36133 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:36133/collection1/
   [junit4]   2> 5732 T42 C0 P36133 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 5733 T42 C0 P36133 oasc.SyncStrategy.syncToMe http://127.0.0.1:36133/collection1/ has no replicas
   [junit4]   2> 5733 T42 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:36133/collection1/ shard1
   [junit4]   2> 5736 T42 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 5762 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 5764 T36 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 5768 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 5859 T12 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 5860 T12 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 5869 T12 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:35108
   [junit4]   2> 5870 T12 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 5872 T12 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 5873 T44 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 5873 T35 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 5873 T12 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1396287267653
   [junit4]   2> 5875 T12 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1396287267653/'
   [junit4]   2> 5906 T12 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1396287267653/solr.xml
   [junit4]   2> 5919 T42 oasc.ZkController.register We are http://127.0.0.1:36133/collection1/ and leader is http://127.0.0.1:36133/collection1/
   [junit4]   2> 5921 T42 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:36133
   [junit4]   2> 5921 T42 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 5921 T42 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 5922 T42 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 5923 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 5924 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 5924 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 5926 T36 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 5928 T36 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:36133",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:36133_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 5933 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 5958 T12 oasc.CoreContainer.<init> New CoreContainer 18241811
   [junit4]   2> 5959 T12 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1396287267653/]
   [junit4]   2> 5960 T12 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 5975 T12 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 5976 T12 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 5976 T12 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 5977 T12 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 5977 T12 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 5978 T12 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 5979 T12 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 5979 T12 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 5980 T12 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 5981 T12 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 5981 T12 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 5982 T12 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41112/solr
   [junit4]   2> 5982 T12 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 5996 T12 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 6004 T55 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1826b name:ZooKeeperConnection Watcher:127.0.0.1:41112 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 6009 T12 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 6035 T12 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 6051 T57 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6eb8f name:ZooKeeperConnection Watcher:127.0.0.1:41112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 6051 T44 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 6051 T12 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 6052 T35 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 6062 T42 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 6068 T12 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 7071 T12 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35108_
   [junit4]   2> 7073 T12 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:35108_
   [junit4]   2> 7077 T57 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 7077 T35 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 7077 T44 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 7090 T58 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 7090 T58 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 7093 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 7094 T58 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 7094 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 7094 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 7095 T36 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 7096 T36 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:35108",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:35108_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 7097 T36 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 7097 T36 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 7102 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 7206 T44 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 7206 T35 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 7206 T57 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 8094 T58 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 8095 T58 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 8095 T58 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 8096 T58 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 8097 T58 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 8097 T58 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1396287267653/collection1/'
   [junit4]   2> 8147 T58 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 8189 T58 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 8191 T58 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 8273 T58 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 8470 T58 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 8472 T58 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 8475 T58 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 8475 T58 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 8477 T58 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 8477 T58 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 8478 T58 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1396287267653/collection1/, dataDir=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty1/
   [junit4]   2> 8478 T58 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 8479 T58 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty1
   [junit4]   2> 8480 T58 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty1/index/
   [junit4]   2> 8480 T58 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 8481 T58 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty1/index
   [junit4]   2> 8485 T58 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@736108 lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty1/index),segFN=segments_1,generation=1}
   [junit4]   2> 8485 T58 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 8487 T58 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 8488 T58 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 8489 T58 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 8489 T58 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 8489 T58 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 8490 T58 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 8490 T58 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 8496 T58 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 8499 T58 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 8501 T58 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 8502 T58 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 8503 T58 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@736108 lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty1/index),segFN=segments_1,generation=1}
   [junit4]   2> 8503 T58 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 8504 T58 oass.SolrIndexSearcher.<init> Opening Searcher@1a0e7db[collection1] main
   [junit4]   2> 8505 T59 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1a0e7db[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 8505 T58 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 8506 T62 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:35108 collection:collection1 shard:shard2
   [junit4]   2> 8507 T12 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0
   [junit4]   2> 8508 T62 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 8508 T12 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 8519 T62 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 8525 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 8525 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 8525 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 8526 T62 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 8526 T62 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C1 name=collection1 org.apache.solr.core.SolrCore@762d0c url=http://127.0.0.1:35108/collection1 node=127.0.0.1:35108_ C1_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:35108, core=collection1, node_name=127.0.0.1:35108_}
   [junit4]   2> 8527 T62 C1 P35108 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:35108/collection1/
   [junit4]   2> 8527 T36 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 8528 T62 C1 P35108 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 8529 T62 C1 P35108 oasc.SyncStrategy.syncToMe http://127.0.0.1:35108/collection1/ has no replicas
   [junit4]   2> 8529 T62 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:35108/collection1/ shard2
   [junit4]   2> 8530 T62 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 8548 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 8557 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 8563 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 8589 T12 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 8590 T12 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 8594 T12 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:37918
   [junit4]   2> 8595 T12 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 8596 T12 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 8596 T12 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1396287270445
   [junit4]   2> 8597 T12 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1396287270445/'
   [junit4]   2> 8617 T12 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1396287270445/solr.xml
   [junit4]   2> 8633 T12 oasc.CoreContainer.<init> New CoreContainer 32713794
   [junit4]   2> 8634 T12 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1396287270445/]
   [junit4]   2> 8635 T12 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 8635 T12 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 8636 T12 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 8636 T12 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 8637 T12 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 8637 T12 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 8638 T12 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 8638 T12 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 8639 T12 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 8640 T12 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 8641 T12 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 8641 T12 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 8642 T12 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41112/solr
   [junit4]   2> 8642 T12 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 8644 T12 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 8647 T73 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1dbbf24 name:ZooKeeperConnection Watcher:127.0.0.1:41112 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 8661 T12 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 8669 T44 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 8669 T35 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 8670 T57 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 8679 T12 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 8682 T75 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16415c0 name:ZooKeeperConnection Watcher:127.0.0.1:41112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 8682 T12 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 8691 T12 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 8708 T62 oasc.ZkController.register We are http://127.0.0.1:35108/collection1/ and leader is http://127.0.0.1:35108/collection1/
   [junit4]   2> 8709 T62 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:35108
   [junit4]   2> 8709 T62 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 8709 T62 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 8710 T62 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 8711 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 8711 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 8712 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 8712 T62 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 8713 T36 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 8714 T36 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:35108",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:35108_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 8716 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 8818 T35 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 8819 T75 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 8819 T57 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 8819 T44 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 9695 T12 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37918_
   [junit4]   2> 9696 T12 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:37918_
   [junit4]   2> 9698 T35 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 9698 T75 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 9698 T57 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 9698 T44 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 9706 T76 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 9706 T76 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 9712 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 9712 T76 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 9712 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 9713 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 9714 T36 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 9715 T36 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:37918",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:37918_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 9715 T36 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 9715 T36 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 9718 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 9822 T35 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 9823 T44 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 9822 T57 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 9822 T75 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 10713 T76 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 10713 T76 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 10714 T76 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 10715 T76 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 10716 T76 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 10716 T76 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1396287270445/collection1/'
   [junit4]   2> 10748 T76 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 10760 T76 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 10762 T76 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 10784 T76 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 10920 T76 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 10923 T76 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 10925 T76 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 10926 T76 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 10929 T76 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 10930 T76 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 10932 T76 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1396287270445/collection1/, dataDir=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty2/
   [junit4]   2> 10932 T76 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 10933 T76 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty2
   [junit4]   2> 10934 T76 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty2/index/
   [junit4]   2> 10934 T76 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 10935 T76 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty2/index
   [junit4]   2> 10944 T76 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@5bfa43 lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty2/index),segFN=segments_1,generation=1}
   [junit4]   2> 10944 T76 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 10946 T76 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 10947 T76 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 10947 T76 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 10947 T76 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 10948 T76 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 10948 T76 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 10949 T76 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 10949 T76 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 10951 T76 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 10961 T76 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 10962 T76 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 10964 T76 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@5bfa43 lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty2/index),segFN=segments_1,generation=1}
   [junit4]   2> 10964 T76 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 10964 T76 oass.SolrIndexSearcher.<init> Opening Searcher@199b755[collection1] main
   [junit4]   2> 10970 T76 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 10972 T77 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@199b755[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 10973 T12 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0
   [junit4]   2> 10976 T12 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 10972 T80 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:37918 collection:collection1 shard:shard1
   [junit4]   2> 10981 T80 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 10996 T80 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 10999 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 11000 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 11000 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 11001 T80 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 11002 T80 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 11002 T36 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> ASYNC  NEW_CORE C2 name=collection1 org.apache.solr.core.SolrCore@1627341 url=http://127.0.0.1:37918/collection1 node=127.0.0.1:37918_ C2_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:37918, core=collection1, node_name=127.0.0.1:37918_}
   [junit4]   2> 11002 T80 C2 P37918 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:37918/collection1/
   [junit4]   2> 11003 T80 C2 P37918 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 11004 T80 C2 P37918 oasc.SyncStrategy.syncToMe http://127.0.0.1:37918/collection1/ has no replicas
   [junit4]   2> 11006 T80 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:37918/collection1/ shard1
   [junit4]   2> 11006 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 11007 T80 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 11015 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 11029 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 11052 T12 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 11053 T12 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 11058 T12 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:53465
   [junit4]   2> 11061 T12 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 11061 T12 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 11061 T12 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1396287272913
   [junit4]   2> 11062 T12 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1396287272913/'
   [junit4]   2> 11082 T12 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0/./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1396287272913/solr.xml
   [junit4]   2> 11100 T12 oasc.CoreContainer.<init> New CoreContainer 2687567
   [junit4]   2> 11101 T12 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1396287272913/]
   [junit4]   2> 11102 T12 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 11102 T12 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 11102 T12 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 11103 T12 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 11103 T12 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 11104 T12 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 11104 T12 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 11105 T12 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 11105 T12 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 11106 T12 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 11106 T12 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 11107 T12 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 11107 T12 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:41112/solr
   [junit4]   2> 11108 T12 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 11109 T12 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 11112 T91 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1437c3a name:ZooKeeperConnection Watcher:127.0.0.1:41112 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 11113 T12 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 11120 T12 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 11124 T93 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@15c1882 name:ZooKeeperConnection Watcher:127.0.0.1:41112/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 11124 T12 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 11132 T57 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 11132 T35 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 11132 T44 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 11132 T75 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 11136 T12 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 11181 T80 oasc.ZkController.register We are http://127.0.0.1:37918/collection1/ and leader is http://127.0.0.1:37918/collection1/
   [junit4]   2> 11182 T80 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:37918
   [junit4]   2> 11182 T80 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 11182 T80 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 11182 T80 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 11184 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 11184 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 11184 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 11185 T80 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 11186 T36 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 11187 T36 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:37918",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:37918_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 11190 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 11292 T75 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 11292 T57 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 11292 T35 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 11292 T93 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 11292 T44 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 12142 T12 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53465_
   [junit4]   2> 12144 T12 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:53465_
   [junit4]   2> 12147 T93 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 12147 T44 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 12147 T57 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 12147 T75 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 12147 T35 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 12152 T94 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 12153 T94 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 12155 T94 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 12155 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 12155 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 12156 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 12157 T36 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 12158 T36 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:53465",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:53465_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 12158 T36 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 12158 T36 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 12161 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 12264 T93 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 12264 T35 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 12264 T75 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 12264 T44 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 12264 T57 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 13155 T94 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 13156 T94 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 13156 T94 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 13157 T94 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 13157 T94 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 13158 T94 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1396287272913/collection1/'
   [junit4]   2> 13175 T94 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 13183 T94 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 13184 T94 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 13200 T94 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 13321 T94 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 13323 T94 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 13326 T94 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 13326 T94 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 13328 T94 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 13328 T94 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 13329 T94 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1396287272913/collection1/, dataDir=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3/
   [junit4]   2> 13329 T94 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 13330 T94 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3
   [junit4]   2> 13331 T94 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3/index/
   [junit4]   2> 13331 T94 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3/index' doesn't exist. Creating new index...
   [junit4]   2> 13331 T94 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3/index
   [junit4]   2> 13333 T94 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@148ac6d lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3/index),segFN=segments_1,generation=1}
   [junit4]   2> 13333 T94 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 13335 T94 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 13335 T94 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 13335 T94 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 13336 T94 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 13336 T94 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 13336 T94 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 13337 T94 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 13337 T94 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 13339 T94 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 13341 T94 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 13341 T94 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 13342 T94 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@148ac6d lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3/index),segFN=segments_1,generation=1}
   [junit4]   2> 13343 T94 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 13343 T94 oass.SolrIndexSearcher.<init> Opening Searcher@1c8c922[collection1] main
   [junit4]   2> 13344 T95 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c8c922[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 13344 T94 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 13345 T98 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:53465 collection:collection1 shard:shard2
   [junit4]   2> 13346 T12 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J0
   [junit4]   2> 13346 T12 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 13347 T12 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 13348 T98 oasc.ZkController.register We are http://127.0.0.1:53465/collection1/ and leader is http://127.0.0.1:35108/collection1/
   [junit4]   2> 13348 T98 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53465
   [junit4]   2> 13349 T98 oasc.ZkController.checkRecovery Core needs to recover:collection1
   [junit4]   2> 13349 T12 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 30 for each attempt
   [junit4]   2> 13349 T98 oasu.DefaultSolrCoreState.doRecovery Running recovery - first canceling any ongoing recovery
   [junit4]   2> 13349 T12 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30
   [junit4]   2> 13350 T12 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> ASYNC  NEW_CORE C3 name=collection1 org.apache.solr.core.SolrCore@bd95b6 url=http://127.0.0.1:53465/collection1 node=127.0.0.1:53465_ C3_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:53465, core=collection1, node_name=127.0.0.1:53465_}
   [junit4]   2> 13354 T99 C3 P53465 oasc.RecoveryStrategy.run Starting recovery process.  core=collection1 recoveringAfterStartup=true
   [junit4]   2> 13354 T98 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 13355 T99 C3 P53465 oasc.RecoveryStrategy.doRecovery ###### startupVersions=[]
   [junit4]   2> 13355 T99 C3 P53465 oasc.ZkController.publish publishing core=collection1 state=recovering collection=collection1
   [junit4]   2> 13355 T99 C3 P53465 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 13357 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 13357 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 13358 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 13359 T36 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 13360 T36 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"recovering",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:53465",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:53465_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node3"}
   [junit4]   2> 13362 T35 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 13465 T93 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 13465 T44 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 13465 T75 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 13465 T35 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 13465 T57 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 13627 T49 oasha.CoreAdminHandler.handleWaitForStateAction Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true
   [junit4]   2> 13634 T49 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 13636 T49 oasha.CoreAdminHandler.handleWaitForStateAction Will wait a max of 183 seconds to see collection1 (shard2 of collection1) have state: recovering
   [junit4]   2> 13637 T49 oasha.CoreAdminHandler.handleWaitForStateAction Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 0 seconds.
   [junit4]   2> 13638 T49 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:53465_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=18 
   [junit4]   2> 14352 T12 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 15354 T12 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2>  C3_STATE=coll:collection1 core:collection1 props:{state=recovering, base_url=http://127.0.0.1:53465, core=collection1, node_name=127.0.0.1:53465_}
   [junit4]   2> 15727 T99 C3 P53465 oasc.RecoveryStrategy.doRecovery Attempting to PeerSync from http://127.0.0.1:35108/collection1/ core=collection1 - recoveringAfterStartup=true
   [junit4]   2> 15734 T99 C3 P53465 oasu.PeerSync.sync PeerSync: core=collection1 url=http://127.0.0.1:53465 START replicas=[http://127.0.0.1:35108/collection1/] nUpdates=100
   [junit4]   2> 15738 T99 C3 P53465 oasu.PeerSync.sync WARN no frame of reference to tell if we've missed updates
   [junit4]   2> 15739 T99 C3 P53465 oasc.RecoveryStrategy.doRecovery PeerSync Recovery was not successful - trying replication. core=collection1
   [junit4]   2> 15740 T99 C3 P53465 oasc.RecoverySt

[...truncated too long message...]

2 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 20046 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
   [junit4]   2> 20047 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
   [junit4]   2> 20047 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
   [junit4]   2> 20047 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
   [junit4]   2> 20048 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
   [junit4]   2> 20048 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
   [junit4]   2> 20048 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
   [junit4]   2> 20049 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
   [junit4]   2> 20049 T12 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
   [junit4]   2> 20051 T12 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 20053 T124 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
   [junit4]   2> 20053 T124 oaz.ClientCnxn$SendThread.run WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable
   [junit4]   2> 	at sun.nio.ch.Net.connect0(Native Method)
   [junit4]   2> 	at sun.nio.ch.Net.connect(Net.java:435)
   [junit4]   2> 	at sun.nio.ch.Net.connect(Net.java:427)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:643)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:967)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   [junit4]   2> 
   [junit4]   2> 20989 T57 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 20989 T57 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 21090 T75 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 21478 T124 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
   [junit4]   2> 21579 T12 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
   [junit4]   2> 21580 T12 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:58260 58260
   [junit4]   2> 21730 T115 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:58260 58260
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 21740 T12 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 21997 T93 C3 P53465 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
   [junit4]   2> 22003 T93 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
   [junit4]   2> 22003 T93 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@bd95b6
   [junit4]   2> 22003 T93 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=354,transaction_logs_total_number=4}
   [junit4]   2> 22004 T93 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 22004 T93 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 22004 T93 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 22007 T93 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 22007 T93 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 22008 T93 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3 [CachedDir<<refCount=0;path=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3;done=false>>]
   [junit4]   2> 22008 T93 oasc.CachingDirectoryFactory.close Closing directory: ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3
   [junit4]   2> 22008 T93 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3/index [CachedDir<<refCount=0;path=./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3/index;done=false>>]
   [junit4]   2> 22008 T93 oasc.CachingDirectoryFactory.close Closing directory: ./solrtest-CloudSolrServerTest-1396287262795/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396287263190/jetty3/index
   [junit4]   2> 22009 T93 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 22009 T93 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 22009 T93 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 22009 T93 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:260)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=Lucene3x, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=es_SV, timezone=Africa/Windhoek
   [junit4]   2> NOTE: Linux 3.8.0-37-generic i386/Oracle Corporation 1.8.0 (32-bit)/cpus=8,threads=1,free=34019568,total=208142336
   [junit4]   2> NOTE: All tests run in this JVM: [CloudSolrServerTest]
   [junit4] Completed on J0 in 23.00s, 2 tests, 1 failure <<< FAILURES!

[...truncated 145 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1276: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:908: There were test failures: 49 suites, 284 tests, 1 failure

Total time: 55 minutes 5 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.8.0 -server -XX:+UseParallelGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure