You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/03/29 04:10:23 UTC

[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0) - Build # 9828 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9828/
Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC

All tests passed

Build Log:
[...truncated 31791 lines...]
-check-forbidden-all:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.7
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.7
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.1
[forbidden-apis] Reading API signatures: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning for API signatures and dependencies...
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.solr.CursorPagingTest (CursorPagingTest.java:971)
[forbidden-apis] Scanned 1973 (and 1362 related) class file(s) for forbidden API invocations (in 1.38s), 1 error(s).

BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:70: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:271: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:479: Check for forbidden API calls failed, see log.

Total time: 60 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_60-ea-b10) - Build # 9830 - Still Failing!

Posted by Robert Muir <rc...@gmail.com>.
thank you!

On Sat, Mar 29, 2014 at 12:52 AM, Shalin Shekhar Mangar
<sh...@apache.org> wrote:
> I committed a fix.
>
> On Sat, Mar 29, 2014 at 12:57 PM, Policeman Jenkins Server
> <je...@thetaphi.de> wrote:
>> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9830/
>> Java: 32bit/jdk1.7.0_60-ea-b10 -client -XX:+UseSerialGC
>>
>> All tests passed
>>
>> Build Log:
>> [...truncated 51534 lines...]
>> BUILD FAILED
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:467: The following error occurred while executing this line:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:406: The following error occurred while executing this line:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:87: The following error occurred while executing this line:
>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:179: The following files are missing svn:eol-style (or binary svn:mime-type):
>> * ./lucene/codecs/src/test/org/apache/lucene/codecs/memory/TestFSTOrdPostingsFormat.java
>> * ./lucene/codecs/src/test/org/apache/lucene/codecs/memory/TestFSTOrdPulsing41PostingsFormat.java
>> * ./lucene/codecs/src/test/org/apache/lucene/codecs/memory/TestFSTPostingsFormat.java
>> * ./lucene/codecs/src/test/org/apache/lucene/codecs/memory/TestFSTPulsing41PostingsFormat.java
>>
>> Total time: 67 minutes 2 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Description set: Java: 32bit/jdk1.7.0_60-ea-b10 -client -XX:+UseSerialGC
>> Archiving artifacts
>> Recording test results
>> Email was triggered for: Failure
>> Sending email for trigger: Failure
>>
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_60-ea-b10) - Build # 9830 - Still Failing!

Posted by Shalin Shekhar Mangar <sh...@apache.org>.
I committed a fix.

On Sat, Mar 29, 2014 at 12:57 PM, Policeman Jenkins Server
<je...@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9830/
> Java: 32bit/jdk1.7.0_60-ea-b10 -client -XX:+UseSerialGC
>
> All tests passed
>
> Build Log:
> [...truncated 51534 lines...]
> BUILD FAILED
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:467: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:406: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:87: The following error occurred while executing this line:
> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:179: The following files are missing svn:eol-style (or binary svn:mime-type):
> * ./lucene/codecs/src/test/org/apache/lucene/codecs/memory/TestFSTOrdPostingsFormat.java
> * ./lucene/codecs/src/test/org/apache/lucene/codecs/memory/TestFSTOrdPulsing41PostingsFormat.java
> * ./lucene/codecs/src/test/org/apache/lucene/codecs/memory/TestFSTPostingsFormat.java
> * ./lucene/codecs/src/test/org/apache/lucene/codecs/memory/TestFSTPulsing41PostingsFormat.java
>
> Total time: 67 minutes 2 seconds
> Build step 'Invoke Ant' marked build as failure
> Description set: Java: 32bit/jdk1.7.0_60-ea-b10 -client -XX:+UseSerialGC
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>



-- 
Regards,
Shalin Shekhar Mangar.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_60-ea-b10) - Build # 9830 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9830/
Java: 32bit/jdk1.7.0_60-ea-b10 -client -XX:+UseSerialGC

All tests passed

Build Log:
[...truncated 51534 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:406: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:87: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:179: The following files are missing svn:eol-style (or binary svn:mime-type):
* ./lucene/codecs/src/test/org/apache/lucene/codecs/memory/TestFSTOrdPostingsFormat.java
* ./lucene/codecs/src/test/org/apache/lucene/codecs/memory/TestFSTOrdPulsing41PostingsFormat.java
* ./lucene/codecs/src/test/org/apache/lucene/codecs/memory/TestFSTPostingsFormat.java
* ./lucene/codecs/src/test/org/apache/lucene/codecs/memory/TestFSTPulsing41PostingsFormat.java

Total time: 67 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 32bit/jdk1.7.0_60-ea-b10 -client -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-4.x-Linux (64bit/jdk1.8.0) - Build # 9829 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/9829/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC

1 tests failed.
REGRESSION:  org.apache.solr.client.solrj.impl.CloudSolrServerTest.testDistribSearch

Error Message:


Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([DE8ADD2A89394F63:5F6C5332FE662F5F]:0)
	at org.junit.Assert.fail(Assert.java:92)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertTrue(Assert.java:54)
	at org.apache.solr.client.solrj.impl.CloudSolrServerTest.doTest(CloudSolrServerTest.java:256)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1617)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:826)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:862)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:876)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:783)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:443)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:835)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:771)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:782)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:359)
	at java.lang.Thread.run(Thread.java:744)




Build Log:
[...truncated 12212 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrServerTest
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421
   [junit4]   2> 22118 T133 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 22118 T133 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 22121 T133 oasc.AbstractZkTestCase.<clinit> WARN TEST_HOME() does not exist - solrj test?
   [junit4]   2> 22126 T133 oas.SolrTestCaseJ4.setUp ###Starting testShutdown
   [junit4]   2> 22129 T133 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 22132 T134 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 22344 T133 oasc.ZkTestServer.run start zk server on port:56191
   [junit4]   2> 22389 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 22552 T140 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@32e1b7e7 name:ZooKeeperConnection Watcher:127.0.0.1:56191 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 22553 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 22555 T133 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 22584 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 22588 T142 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45a96b47 name:ZooKeeperConnection Watcher:127.0.0.1:56191/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 22588 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 22594 T133 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 22599 T133 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 22604 T133 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 22606 T133 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 22610 T133 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 22611 T133 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 22616 T133 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 22616 T133 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 22620 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
   [junit4]   2> 22620 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
   [junit4]   2> 22621 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
   [junit4]   2> 22621 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
   [junit4]   2> 22621 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
   [junit4]   2> 22622 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
   [junit4]   2> 22622 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
   [junit4]   2> 22622 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
   [junit4]   2> 22623 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
   [junit4]   2> 22632 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 22633 T143 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
   [junit4]   2> 22634 T143 oaz.ClientCnxn$SendThread.run WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Network is unreachable
   [junit4]   2> 	at sun.nio.ch.Net.connect0(Native Method)
   [junit4]   2> 	at sun.nio.ch.Net.connect(Net.java:435)
   [junit4]   2> 	at sun.nio.ch.Net.connect(Net.java:427)
   [junit4]   2> 	at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:643)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.registerAndConnect(ClientCnxnSocketNIO.java:277)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.connect(ClientCnxnSocketNIO.java:287)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:967)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1003)
   [junit4]   2> 
   [junit4]   2> 23736 T143 oaz.ClientCnxnSocketNIO.connect ERROR Unable to open socket to ff01::114/ff01:0:0:0:0:0:0:114:33332
   [junit4]   2> 23839 T133 oas.SolrTestCaseJ4.tearDown ###Ending testShutdown
   [junit4]   2> 23840 T133 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:56191 56191
   [junit4]   2> 23941 T134 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:56191 56191
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 23947 T133 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 23947 T133 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 23948 T145 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 24048 T133 oasc.ZkTestServer.run start zk server on port:48823
   [junit4]   2> 24049 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 24052 T151 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7f153e9a name:ZooKeeperConnection Watcher:127.0.0.1:48823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 24052 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 24053 T133 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 24060 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 24062 T153 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5f47355e name:ZooKeeperConnection Watcher:127.0.0.1:48823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 24062 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 24062 T133 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 24069 T133 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 24073 T133 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 24075 T133 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 24078 T133 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 24079 T133 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 24083 T133 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 24084 T133 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 24087 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml because it doesn't exist
   [junit4]   2> 24087 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/stopwords.txt because it doesn't exist
   [junit4]   2> 24087 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/protwords.txt because it doesn't exist
   [junit4]   2> 24088 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/currency.xml because it doesn't exist
   [junit4]   2> 24088 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/enumsConfig.xml because it doesn't exist
   [junit4]   2> 24088 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/open-exchange-rates.json because it doesn't exist
   [junit4]   2> 24088 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/mapping-ISOLatin1Accent.txt because it doesn't exist
   [junit4]   2> 24089 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/old_synonyms.txt because it doesn't exist
   [junit4]   2> 24089 T133 oasc.AbstractZkTestCase.putConfig skipping /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/solrj/src/test-files/solrj/solr/collection1/conf/synonyms.txt because it doesn't exist
   [junit4]   2> 24145 T133 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 24148 T133 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:40430
   [junit4]   2> 24149 T133 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 24149 T133 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 24149 T133 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1396069839394
   [junit4]   2> 24150 T133 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1396069839394/'
   [junit4]   2> 24159 T133 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1396069839394/solr.xml
   [junit4]   2> 24170 T133 oasc.CoreContainer.<init> New CoreContainer 280825532
   [junit4]   2> 24170 T133 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1396069839394/]
   [junit4]   2> 24171 T133 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 24171 T133 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 24171 T133 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 24172 T133 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 24172 T133 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 24172 T133 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 24173 T133 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 24173 T133 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 24173 T133 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 24174 T133 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 24174 T133 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 24175 T133 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 24175 T133 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48823/solr
   [junit4]   2> 24188 T133 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 24189 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 24192 T164 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cb155a2 name:ZooKeeperConnection Watcher:127.0.0.1:48823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 24192 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 24197 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 24199 T166 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3400b2a0 name:ZooKeeperConnection Watcher:127.0.0.1:48823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 24199 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 24205 T133 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 24210 T133 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 24214 T133 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 24218 T133 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 24221 T133 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 24225 T133 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 24227 T133 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40430_
   [junit4]   2> 24229 T133 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:40430_
   [junit4]   2> 24232 T133 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 24234 T133 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 24245 T133 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:40430_
   [junit4]   2> 24245 T133 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 24248 T133 oasc.Overseer.start Overseer (id=91492832985350147-127.0.0.1:40430_-n_0000000000) starting
   [junit4]   2> 24258 T133 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 24287 T168 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 24287 T133 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 24290 T133 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 24292 T133 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 24295 T168 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 24308 T167 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 24314 T169 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 24314 T169 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 24318 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 24318 T169 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 24320 T167 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 24321 T167 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:40430",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:40430_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 24322 T167 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 24330 T167 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 24335 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 24338 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 25319 T169 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 25319 T169 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 25320 T169 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 25320 T169 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 25321 T169 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 25322 T169 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1396069839394/collection1/'
   [junit4]   2> 25344 T169 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 25355 T169 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 25357 T169 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 25370 T169 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 25518 T169 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 25520 T169 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 25521 T169 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 25522 T169 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 25522 T169 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 25523 T169 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 25523 T169 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-controljetty-1396069839394/collection1/, dataDir=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/control/data/
   [junit4]   2> 25523 T169 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 25524 T169 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/control/data
   [junit4]   2> 25524 T169 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/control/data/index/
   [junit4]   2> 25524 T169 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 25525 T169 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/control/data/index
   [junit4]   2> 25527 T169 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@125294b3 lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/control/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 25527 T169 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 25528 T169 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 25528 T169 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 25529 T169 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 25529 T169 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 25529 T169 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 25530 T169 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 25530 T169 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 25530 T169 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 25532 T169 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 25534 T169 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 25534 T169 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 25535 T169 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@125294b3 lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/control/data/index),segFN=segments_1,generation=1}
   [junit4]   2> 25535 T169 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 25535 T169 oass.SolrIndexSearcher.<init> Opening Searcher@6a96c533[collection1] main
   [junit4]   2> 25536 T170 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6a96c533[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 25536 T169 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 25537 T173 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:40430 collection:control_collection shard:shard1
   [junit4]   2> 25538 T133 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1
   [junit4]   2> 25538 T133 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 25539 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 25542 T175 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6a90b83a name:ZooKeeperConnection Watcher:127.0.0.1:48823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 25543 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 25544 T173 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 25544 T133 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 25549 T133 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 25553 T173 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 25555 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 25556 T173 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 25557 T173 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 25557 T167 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> ASYNC  NEW_CORE C28 name=collection1 org.apache.solr.core.SolrCore@2e8a8885 url=http://127.0.0.1:40430/collection1 node=127.0.0.1:40430_ C28_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:40430, core=collection1, node_name=127.0.0.1:40430_}
   [junit4]   2> 25558 T173 C28 P40430 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:40430/collection1/
   [junit4]   2> 25560 T173 C28 P40430 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 25563 T173 C28 P40430 oasc.SyncStrategy.syncToMe http://127.0.0.1:40430/collection1/ has no replicas
   [junit4]   2> 25563 T173 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:40430/collection1/ shard1
   [junit4]   2> 25563 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 25565 T173 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 25573 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 25577 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 25605 T133 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 25605 T133 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 25607 T133 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:52442
   [junit4]   2> 25608 T133 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 25609 T133 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 25609 T133 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1396069840853
   [junit4]   2> 25609 T133 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1396069840853/'
   [junit4]   2> 25622 T133 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1396069840853/solr.xml
   [junit4]   2> 25638 T133 oasc.CoreContainer.<init> New CoreContainer 1149096847
   [junit4]   2> 25638 T133 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1396069840853/]
   [junit4]   2> 25639 T133 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 25639 T133 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 25639 T133 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 25639 T133 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 25639 T133 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 25639 T133 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 25640 T133 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 25640 T133 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 25640 T133 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 25641 T133 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 25641 T133 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 25641 T133 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 25642 T133 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48823/solr
   [junit4]   2> 25642 T133 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 25643 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 25645 T186 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67b0c2a0 name:ZooKeeperConnection Watcher:127.0.0.1:48823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 25645 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 25648 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 25651 T188 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@75933224 name:ZooKeeperConnection Watcher:127.0.0.1:48823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 25651 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 25662 T133 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 25679 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 25679 T175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 25679 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 25724 T173 oasc.ZkController.register We are http://127.0.0.1:40430/collection1/ and leader is http://127.0.0.1:40430/collection1/
   [junit4]   2> 25725 T173 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:40430
   [junit4]   2> 25725 T173 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 25725 T173 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 25725 T173 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 25727 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 25727 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 25727 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 25727 T173 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 25729 T167 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 25730 T167 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:40430",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:40430_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 25732 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 25835 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 25835 T175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 25835 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 26665 T133 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52442_
   [junit4]   2> 26667 T133 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52442_
   [junit4]   2> 26669 T166 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 26669 T175 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 26669 T188 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 26676 T189 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 26676 T189 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 26677 T189 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 26677 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 26678 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 26678 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 26680 T167 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 26681 T167 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52442",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52442_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 26681 T167 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 26681 T167 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 26687 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 26789 T175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 26789 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 26790 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 27678 T189 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 27678 T189 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 27679 T189 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 27679 T189 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 27680 T189 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 27680 T189 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1396069840853/collection1/'
   [junit4]   2> 27697 T189 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 27711 T189 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 27713 T189 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 27739 T189 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 27896 T189 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 27897 T189 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 27899 T189 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 27899 T189 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 27900 T189 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 27900 T189 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 27900 T189 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty1-1396069840853/collection1/, dataDir=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty1/
   [junit4]   2> 27900 T189 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 27901 T189 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty1
   [junit4]   2> 27901 T189 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty1/index/
   [junit4]   2> 27901 T189 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 27902 T189 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty1/index
   [junit4]   2> 27903 T189 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6f1145c2 lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty1/index),segFN=segments_1,generation=1}
   [junit4]   2> 27904 T189 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 27905 T189 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 27905 T189 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 27905 T189 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 27905 T189 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 27905 T189 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 27906 T189 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 27906 T189 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 27906 T189 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 27907 T189 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 27909 T189 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 27909 T189 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 27909 T189 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6f1145c2 lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty1/index),segFN=segments_1,generation=1}
   [junit4]   2> 27910 T189 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 27910 T189 oass.SolrIndexSearcher.<init> Opening Searcher@6101eca7[collection1] main
   [junit4]   2> 27911 T189 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 27911 T190 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6101eca7[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 27911 T193 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52442 collection:collection1 shard:shard2
   [junit4]   2> 27912 T133 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1
   [junit4]   2> 27912 T133 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 27919 T193 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 27926 T193 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 27929 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 27929 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 27929 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 27929 T193 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 27929 T193 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C29 name=collection1 org.apache.solr.core.SolrCore@35bc9ea5 url=http://127.0.0.1:52442/collection1 node=127.0.0.1:52442_ C29_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:52442, core=collection1, node_name=127.0.0.1:52442_}
   [junit4]   2> 27930 T193 C29 P52442 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52442/collection1/
   [junit4]   2> 27930 T193 C29 P52442 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 27930 T167 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 27930 T193 C29 P52442 oasc.SyncStrategy.syncToMe http://127.0.0.1:52442/collection1/ has no replicas
   [junit4]   2> 27930 T193 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52442/collection1/ shard2
   [junit4]   2> 27931 T193 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 27934 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 27937 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 27941 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 27961 T133 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 27962 T133 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 27972 T133 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:45459
   [junit4]   2> 27973 T133 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 27973 T133 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 27973 T133 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1396069843217
   [junit4]   2> 27973 T133 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1396069843217/'
   [junit4]   2> 27983 T133 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1396069843217/solr.xml
   [junit4]   2> 28006 T133 oasc.CoreContainer.<init> New CoreContainer 1524244302
   [junit4]   2> 28006 T133 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1396069843217/]
   [junit4]   2> 28007 T133 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 28007 T133 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 28007 T133 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 28008 T133 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 28008 T133 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 28008 T133 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 28009 T133 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 28009 T133 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 28009 T133 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 28010 T133 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 28010 T133 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 28010 T133 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 28011 T133 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48823/solr
   [junit4]   2> 28011 T133 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 28012 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 28014 T204 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@fef468b name:ZooKeeperConnection Watcher:127.0.0.1:48823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 28015 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 28017 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 28019 T206 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@191d755b name:ZooKeeperConnection Watcher:127.0.0.1:48823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 28019 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 28033 T133 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 28043 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 28044 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 28043 T175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 28044 T206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 28088 T193 oasc.ZkController.register We are http://127.0.0.1:52442/collection1/ and leader is http://127.0.0.1:52442/collection1/
   [junit4]   2> 28088 T193 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52442
   [junit4]   2> 28088 T193 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 28088 T193 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 28088 T193 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 28089 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 28090 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 28090 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 28090 T193 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 28091 T167 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 28092 T167 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52442",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52442_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 28094 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 28197 T175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 28197 T206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 28197 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 28197 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 29036 T133 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45459_
   [junit4]   2> 29037 T133 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:45459_
   [junit4]   2> 29039 T175 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 29039 T206 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 29039 T166 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 29039 T188 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 29046 T207 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 29046 T207 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 29048 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 29048 T207 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 29048 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 29048 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 29049 T167 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 29050 T167 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:45459",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:45459_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 29050 T167 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 29050 T167 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 29054 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 29156 T175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 29156 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 29156 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 29156 T206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 30048 T207 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 30048 T207 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 30050 T207 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 30050 T207 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 30052 T207 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 30052 T207 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1396069843217/collection1/'
   [junit4]   2> 30083 T207 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 30101 T207 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 30103 T207 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 30117 T207 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 30268 T207 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 30269 T207 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 30271 T207 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 30272 T207 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 30272 T207 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 30273 T207 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 30273 T207 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty2-1396069843217/collection1/, dataDir=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2/
   [junit4]   2> 30273 T207 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 30274 T207 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2
   [junit4]   2> 30275 T207 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2/index/
   [junit4]   2> 30275 T207 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 30275 T207 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2/index
   [junit4]   2> 30277 T207 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2f411086 lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2/index),segFN=segments_1,generation=1}
   [junit4]   2> 30277 T207 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 30278 T207 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 30279 T207 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 30279 T207 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 30286 T207 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 30287 T207 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 30287 T207 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 30287 T207 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 30288 T207 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 30290 T207 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 30291 T207 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 30291 T207 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 30292 T207 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2f411086 lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2/index),segFN=segments_1,generation=1}
   [junit4]   2> 30292 T207 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 30293 T207 oass.SolrIndexSearcher.<init> Opening Searcher@39fd7686[collection1] main
   [junit4]   2> 30294 T208 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@39fd7686[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 30294 T207 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 30295 T211 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:45459 collection:collection1 shard:shard1
   [junit4]   2> 30296 T133 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1
   [junit4]   2> 30296 T133 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 30300 T211 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 30323 T211 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 30326 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 30326 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 30326 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 30327 T211 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 30327 T211 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C30 name=collection1 org.apache.solr.core.SolrCore@398a14fc url=http://127.0.0.1:45459/collection1 node=127.0.0.1:45459_ C30_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:45459, core=collection1, node_name=127.0.0.1:45459_}
   [junit4]   2> 30327 T211 C30 P45459 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:45459/collection1/
   [junit4]   2> 30327 T211 C30 P45459 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 30327 T211 C30 P45459 oasc.SyncStrategy.syncToMe http://127.0.0.1:45459/collection1/ has no replicas
   [junit4]   2> 30327 T167 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 30328 T211 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:45459/collection1/ shard1
   [junit4]   2> 30328 T211 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 30331 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 30334 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 30339 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 30378 T133 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 30379 T133 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 30402 T133 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:34004
   [junit4]   2> 30402 T133 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 30403 T133 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 30403 T133 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1396069845601
   [junit4]   2> 30403 T133 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1396069845601/'
   [junit4]   2> 30420 T133 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1396069845601/solr.xml
   [junit4]   2> 30433 T133 oasc.CoreContainer.<init> New CoreContainer 346615697
   [junit4]   2> 30433 T133 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1396069845601/]
   [junit4]   2> 30434 T133 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 30434 T133 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 30435 T133 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 30435 T133 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 30435 T133 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 30435 T133 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 30436 T133 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 30436 T133 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 30436 T133 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 30437 T133 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 30437 T133 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 30438 T133 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 30438 T133 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:48823/solr
   [junit4]   2> 30438 T133 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 30439 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 30442 T222 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3f260b86 name:ZooKeeperConnection Watcher:127.0.0.1:48823 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 30442 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 30476 T175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 30476 T206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 30476 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 30476 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 30480 T133 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 30490 T224 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68fe36c1 name:ZooKeeperConnection Watcher:127.0.0.1:48823/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 30490 T211 oasc.ZkController.register We are http://127.0.0.1:45459/collection1/ and leader is http://127.0.0.1:45459/collection1/
   [junit4]   2> 30490 T133 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 30490 T211 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45459
   [junit4]   2> 30491 T211 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 30491 T211 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 30492 T211 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 30493 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 30493 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 30493 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 30496 T211 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 30498 T167 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 30499 T167 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:45459",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:45459_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 30502 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 30504 T133 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 30604 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 30605 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 30604 T224 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 30604 T175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 30608 T206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 31507 T133 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34004_
   [junit4]   2> 31509 T133 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:34004_
   [junit4]   2> 31511 T188 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 31511 T206 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 31512 T224 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 31512 T166 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 31512 T175 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 31537 T225 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 31537 T225 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 31618 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 31618 T225 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 31618 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 31618 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 31620 T167 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 31621 T167 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:34004",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:34004_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 31621 T167 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 31621 T167 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 31626 T166 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 31729 T206 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 31729 T224 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 31729 T166 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 31729 T188 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 31729 T175 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 4)
   [junit4]   2> 32618 T225 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 32618 T225 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 32619 T225 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 32619 T225 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 32620 T225 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 32620 T225 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1396069845601/collection1/'
   [junit4]   2> 32642 T225 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 32649 T225 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 32651 T225 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 32656 T225 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 32786 T225 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 32788 T225 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 32790 T225 oass.IndexSchema.readSchema WARN Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
   [junit4]   2> 32790 T225 oass.IndexSchema.readSchema WARN Field text is not multivalued and destination for multiple copyFields (3)
   [junit4]   2> 32791 T225 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 32791 T225 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 32792 T225 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build/solr-solrj/test/J1/./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-jetty3-1396069845601/collection1/, dataDir=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3/
   [junit4]   2> 32792 T225 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 32792 T225 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3
   [junit4]   2> 32793 T225 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3/index/
   [junit4]   2> 32793 T225 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3/index' doesn't exist. Creating new index...
   [junit4]   2> 32795 T225 oasc.CachingDirectoryFactory.get return new directory for ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3/index
   [junit4]   2> 32796 T225 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2f19626b lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3/index),segFN=segments_1,generation=1}
   [junit4]   2> 32796 T225 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 32798 T225 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 32798 T225 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 32798 T225 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 32798 T225 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 32798 T225 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 32799 T225 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 32799 T225 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 32799 T225 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: solr.PingRequestHandler
   [junit4]   2> 32801 T225 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 32802 T225 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 32802 T225 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 32803 T225 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@2f19626b lockFactory=NativeFSLockFactory@./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3/index),segFN=segments_1,generation=1}
   [junit4]   2> 32803 T225 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 32803 T225 oass.SolrIndexSearcher.<init> Opening Searcher@204b6e9c[collection1] main
   [junit4]   2> 32804 T225 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 32804 T226 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@204b6e9c[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 32804 T229 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:34004 collection:collection1 shard:shard2
   [junit4]   2> 32805 T133 oass.SolrDispatch

[...truncated too long message...]

> 39342 T133 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2/index [CachedDir<<refCount=0;path=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2/index;done=false>>]
   [junit4]   2> 39342 T133 oasc.CachingDirectoryFactory.close Closing directory: ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2/index
   [junit4]   2> 39342 T133 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2 [CachedDir<<refCount=0;path=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2;done=false>>]
   [junit4]   2> 39342 T206 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 39342 T133 oasc.CachingDirectoryFactory.close Closing directory: ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty2
   [junit4]   2> 39367 T133 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 39419 T133 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 34004
   [junit4]   2> 39419 T133 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=346615697
   [junit4]   2> 39419 T133 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 39419 T133 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 39445 T133 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 39497 T133 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 39498 T133 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:48823 48823
   [junit4]   2> 39696 T145 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:48823 48823
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CloudSolrServerTest -Dtests.method=testDistribSearch -Dtests.seed=DE8ADD2A89394F63 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar -Dtests.timezone=Australia/Adelaide -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 15.8s J1 | CloudSolrServerTest.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DE8ADD2A89394F63:5F6C5332FE662F5F]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrServerTest.doTest(CloudSolrServerTest.java:256)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:867)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:744)
   [junit4]   2> 39747 T133 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 40757 T188 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 40758 T188 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 40845 T206 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> ASYNC  NEW_CORE C35 name=collection1 org.apache.solr.core.SolrCore@5c377aff url=http://127.0.0.1:34004/collection1 node=127.0.0.1:34004_ C35_STATE=coll:collection1 core:collection1 props:{state=active, base_url=http://127.0.0.1:34004, core=collection1, node_name=127.0.0.1:34004_}
   [junit4]   2> 41765 T224 C35 P34004 oasc.SyncStrategy.sync WARN Closed, skipping sync up.
   [junit4]   2> 41767 T224 oasc.ShardLeaderElectionContext.rejoinLeaderElection Not rejoining election because CoreContainer is shutdown
   [junit4]   2> 41767 T224 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@5c377aff
   [junit4]   2> 41767 T224 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=5,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=2,cumulative_deletesById=1,cumulative_deletesByQuery=1,cumulative_errors=0,transaction_logs_total_size=354,transaction_logs_total_number=4}
   [junit4]   2> 41768 T224 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 41768 T224 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 41768 T224 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 41770 T224 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 41770 T224 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 41770 T224 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3 [CachedDir<<refCount=0;path=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3;done=false>>]
   [junit4]   2> 41771 T224 oasc.CachingDirectoryFactory.close Closing directory: ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3
   [junit4]   2> 41771 T224 oasc.CachingDirectoryFactory.closeCacheValue looking to close ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3/index [CachedDir<<refCount=0;path=./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3/index;done=false>>]
   [junit4]   2> 41771 T224 oasc.CachingDirectoryFactory.close Closing directory: ./solrtest-CloudSolrServerTest-1396069837421/org.apache.solr.client.solrj.impl.CloudSolrServerTest-1396069839251/jetty3/index
   [junit4]   2> 41771 T224 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 41771 T224 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 41772 T224 oascc.ZkStateReader$2.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 41772 T224 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:259)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:256)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:256)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=Lucene3x, sim=RandomSimilarityProvider(queryNorm=true,coord=crazy): {}, locale=ar, timezone=Australia/Adelaide
   [junit4]   2> NOTE: Linux 3.8.0-37-generic amd64/Oracle Corporation 1.8.0 (64-bit)/cpus=8,threads=1,free=146395880,total=189792256
   [junit4]   2> NOTE: All tests run in this JVM: [SolrDocumentTest, MergeIndexesEmbeddedTest, SolrExampleJettyTest, TestHash, NamedListTest, SolrPingTest, MultiCoreEmbeddedTest, IteratorChainTest, MultiCoreExampleJettyTest, TestBatchUpdate, FacetFieldTest, CloudSolrServerTest]
   [junit4] Completed on J1 in 20.78s, 2 tests, 1 failure <<< FAILURES!

[...truncated 82 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:467: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:447: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:45: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/build.xml:202: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/solr/common-build.xml:496: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1276: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:908: There were test failures: 49 suites, 284 tests, 1 failure

Total time: 58 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Description set: Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure