You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/02/23 08:29:41 UTC

[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1909 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1909/

All tests passed

Build Log:
[...truncated 28657 lines...]
check-licenses:
     [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jcl-over-slf4j-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jcl-over-slf4j-1.6.6.jar.sha1

 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jul-to-slf4j-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jul-to-slf4j-1.6.6.jar.sha1
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/slf4j-api-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/slf4j-api-1.6.6.jar.sha1

[...truncated 3 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:459: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:64: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.

Total time: 108 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1916 - Still Failing

Posted by Robert Muir <rc...@gmail.com>.
Thanks Mark!

On Sun, Mar 2, 2014 at 11:45 AM, Mark Miller <ma...@gmail.com> wrote:
> I just tried zapping the workspace for that job.
>
> - Mark
>
> http://about.me/markrmiller
>
> On Mar 2, 2014, at 11:44 AM, Mark Miller <ma...@gmail.com> wrote:
>
>> Yeah, I was just trying to figure this out too - best I can tell, that check out is in a bad state.
>>
>> - Mark
>>
>> http://about.me/markrmiller
>>
>> On Mar 2, 2014, at 11:14 AM, Robert Muir <rc...@gmail.com> wrote:
>>
>>> I think this one needs a clean-jars or similar on the workspace?
>>>
>>>
>>> On Sun, Mar 2, 2014 at 11:11 AM, Apache Jenkins Server
>>> <je...@builds.apache.org> wrote:
>>>> Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1916/
>>>>
>>>> All tests passed
>>>>
>>>> Build Log:
>>>> [...truncated 28344 lines...]
>>>> check-licenses:
>>>>    [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
>>>> [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jcl-over-slf4j-1.6.6.jar
>>>> [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jcl-over-slf4j-1.6.6.jar.sha1
>>>>
>>>> [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jul-to-slf4j-1.6.6.jar
>>>> [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jul-to-slf4j-1.6.6.jar.sha1
>>>> [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/slf4j-api-1.6.6.jar
>>>> [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/slf4j-api-1.6.6.jar.sha1
>>>>
>>>> [...truncated 3 lines...]
>>>> BUILD FAILED
>>>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:471: The following error occurred while executing this line:
>>>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:64: The following error occurred while executing this line:
>>>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
>>>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.
>>>>
>>>> Total time: 111 minutes 2 seconds
>>>> Build step 'Invoke Ant' marked build as failure
>>>> Archiving artifacts
>>>> Recording test results
>>>> Email was triggered for: Failure
>>>> Sending email for trigger: Failure
>>>>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1916 - Still Failing

Posted by Mark Miller <ma...@gmail.com>.
I just tried zapping the workspace for that job.

- Mark

http://about.me/markrmiller

On Mar 2, 2014, at 11:44 AM, Mark Miller <ma...@gmail.com> wrote:

> Yeah, I was just trying to figure this out too - best I can tell, that check out is in a bad state.
> 
> - Mark
> 
> http://about.me/markrmiller
> 
> On Mar 2, 2014, at 11:14 AM, Robert Muir <rc...@gmail.com> wrote:
> 
>> I think this one needs a clean-jars or similar on the workspace?
>> 
>> 
>> On Sun, Mar 2, 2014 at 11:11 AM, Apache Jenkins Server
>> <je...@builds.apache.org> wrote:
>>> Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1916/
>>> 
>>> All tests passed
>>> 
>>> Build Log:
>>> [...truncated 28344 lines...]
>>> check-licenses:
>>>    [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
>>> [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jcl-over-slf4j-1.6.6.jar
>>> [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jcl-over-slf4j-1.6.6.jar.sha1
>>> 
>>> [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jul-to-slf4j-1.6.6.jar
>>> [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jul-to-slf4j-1.6.6.jar.sha1
>>> [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/slf4j-api-1.6.6.jar
>>> [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/slf4j-api-1.6.6.jar.sha1
>>> 
>>> [...truncated 3 lines...]
>>> BUILD FAILED
>>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:471: The following error occurred while executing this line:
>>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:64: The following error occurred while executing this line:
>>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
>>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.
>>> 
>>> Total time: 111 minutes 2 seconds
>>> Build step 'Invoke Ant' marked build as failure
>>> Archiving artifacts
>>> Recording test results
>>> Email was triggered for: Failure
>>> Sending email for trigger: Failure
>>> 
>>> 
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1916 - Still Failing

Posted by Mark Miller <ma...@gmail.com>.
Yeah, I was just trying to figure this out too - best I can tell, that check out is in a bad state.

- Mark

http://about.me/markrmiller

On Mar 2, 2014, at 11:14 AM, Robert Muir <rc...@gmail.com> wrote:

> I think this one needs a clean-jars or similar on the workspace?
> 
> 
> On Sun, Mar 2, 2014 at 11:11 AM, Apache Jenkins Server
> <je...@builds.apache.org> wrote:
>> Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1916/
>> 
>> All tests passed
>> 
>> Build Log:
>> [...truncated 28344 lines...]
>> check-licenses:
>>     [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
>> [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jcl-over-slf4j-1.6.6.jar
>> [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jcl-over-slf4j-1.6.6.jar.sha1
>> 
>> [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jul-to-slf4j-1.6.6.jar
>> [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jul-to-slf4j-1.6.6.jar.sha1
>> [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/slf4j-api-1.6.6.jar
>> [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/slf4j-api-1.6.6.jar.sha1
>> 
>> [...truncated 3 lines...]
>> BUILD FAILED
>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:471: The following error occurred while executing this line:
>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:64: The following error occurred while executing this line:
>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
>> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.
>> 
>> Total time: 111 minutes 2 seconds
>> Build step 'Invoke Ant' marked build as failure
>> Archiving artifacts
>> Recording test results
>> Email was triggered for: Failure
>> Sending email for trigger: Failure
>> 
>> 
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1916 - Still Failing

Posted by Robert Muir <rc...@gmail.com>.
I think this one needs a clean-jars or similar on the workspace?


On Sun, Mar 2, 2014 at 11:11 AM, Apache Jenkins Server
<je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1916/
>
> All tests passed
>
> Build Log:
> [...truncated 28344 lines...]
> check-licenses:
>      [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
>  [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jcl-over-slf4j-1.6.6.jar
>  [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jcl-over-slf4j-1.6.6.jar.sha1
>
>  [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jul-to-slf4j-1.6.6.jar
>  [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jul-to-slf4j-1.6.6.jar.sha1
>  [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/slf4j-api-1.6.6.jar
>  [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/slf4j-api-1.6.6.jar.sha1
>
> [...truncated 3 lines...]
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:471: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:64: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.
>
> Total time: 111 minutes 2 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1916 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1916/

All tests passed

Build Log:
[...truncated 28344 lines...]
check-licenses:
     [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jcl-over-slf4j-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jcl-over-slf4j-1.6.6.jar.sha1

 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jul-to-slf4j-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jul-to-slf4j-1.6.6.jar.sha1
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/slf4j-api-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/slf4j-api-1.6.6.jar.sha1

[...truncated 3 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:471: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:64: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.

Total time: 111 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1915 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1915/

All tests passed

Build Log:
[...truncated 28367 lines...]
check-licenses:
     [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jcl-over-slf4j-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jcl-over-slf4j-1.6.6.jar.sha1

 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jul-to-slf4j-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jul-to-slf4j-1.6.6.jar.sha1
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/slf4j-api-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/slf4j-api-1.6.6.jar.sha1

[...truncated 3 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:471: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:64: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.

Total time: 103 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1914 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1914/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.OverseerTest.testOverseerFailure

Error Message:
KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1

Stack Trace:
org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
	at __randomizedtesting.SeedInfo.seed([9F02B25F17FA0C85:9B0A3DAC055FE3A4]:0)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
	at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:429)
	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:426)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:383)
	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:370)
	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:156)
	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:289)
	at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:154)
	at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:590)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:724)




Build Log:
[...truncated 10218 lines...]
   [junit4] Suite: org.apache.solr.cloud.OverseerTest
   [junit4]   2> 127058 T368 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 127059 T368 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solrtest-OverseerTest-1393611306913
   [junit4]   2> 127060 T368 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 127066 T368 oas.SolrTestCaseJ4.setUp ###Starting testOverseerFailure
   [junit4]   2> 127067 T368 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 127067 T369 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 127168 T368 oasc.ZkTestServer.run start zk server on port:14056
   [junit4]   2> 127169 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 127201 T375 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@129ace22 name:ZooKeeperConnection Watcher:127.0.0.1:14056/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 127201 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 127202 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 127205 T377 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@365d1fe2 name:ZooKeeperConnection Watcher:127.0.0.1:14056 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 127206 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 127215 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 127218 T379 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@267e5d35 name:ZooKeeperConnection Watcher:127.0.0.1:14056 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 127219 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 127219 T368 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 127223 T368 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 127233 T368 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 127236 T368 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 127238 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 127242 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 127244 T381 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@21065fec name:ZooKeeperConnection Watcher:127.0.0.1:14056/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 127245 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 127246 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 127249 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
   [junit4]   2> 127252 T381 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 127252 T375 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 127253 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 127255 T383 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a0a925d name:ZooKeeperConnection Watcher:127.0.0.1:14056/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 127255 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 127257 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 127259 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 127271 T368 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:14056_solr
   [junit4]   2> 127272 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 127275 T368 oasc.Overseer.start Overseer (id=91331710610964484-127.0.0.1:14056_solr-n_0000000000) starting
   [junit4]   2> 127284 T368 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 127288 T368 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 127292 T368 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 127296 T385 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 127297 T385 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 127297 T384 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 128299 T383 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 128301 T384 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 128302 T384 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"recovering",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 128303 T384 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1]
   [junit4]   2> 128303 T384 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 128312 T383 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 128314 T381 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 128314 T375 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 128800 T368 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 128828 T368 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 128835 T383 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 128836 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 128837 T384 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 128840 T383 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 128840 T368 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 128842 T384 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 128843 T384 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 128846 T383 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 128949 T375 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 128951 T381 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 130008 T368 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 130036 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130039 T387 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4befe1a1 name:ZooKeeperConnection Watcher:127.0.0.1:14056/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130039 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130051 T368 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:14056_solr
   [junit4]   2> 130051 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 130054 T368 oasc.Overseer.start Overseer (id=91331710610964485-127.0.0.1:14056_solr-n_0000000001) starting
   [junit4]   2> 130071 T389 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 130072 T389 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 130072 T388 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 130075 T388 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 130076 T388 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"recovering",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 130076 T388 oasc.Overseer$ClusterStateUpdater.updateState shard=shard1 is already registered
   [junit4]   2> 130080 T387 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 130081 T381 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 130081 T375 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 130137 T387 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 130137 T368 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 130139 T388 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 130152 T368 oasc.LeaderElector.checkIfIamLeader WARN Failed setting watch org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /collections/collection1/leader_elect/shard1/election/91331710610964483-node1_core1-n_0000000002
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1151)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:271)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:271)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:125)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:289)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:154)
   [junit4]   2> 	at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:590)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:606)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:724)
   [junit4]   2> 
   [junit4]   2> 130161 T381 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 130163 T388 oasc.SolrException.log ERROR Problem cleaning up collection in zk:collection1:org.apache.zookeeper.KeeperException$NotEmptyException: KeeperErrorCode = Directory not empty for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:125)
   [junit4]   2> 		at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 		at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:174)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient$2.execute(SolrZkClient.java:171)
   [junit4]   2> 		at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.delete(SolrZkClient.java:171)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:633)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:628)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:628)
   [junit4]   2> 		at org.apache.solr.common.cloud.SolrZkClient.clean(SolrZkClient.java:628)
   [junit4]   2> 		at org.apache.solr.cloud.Overseer$ClusterStateUpdater.removeCore(Overseer.java:949)
   [junit4]   2> 		at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:235)
   [junit4]   2> 		at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:192)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:724)
   [junit4]   2> 	
   [junit4]   2> 130163 T368 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 130174 T387 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 130177 T387 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 130177 T375 oascc.ZkStateReader$3.process Updating live nodes... (0)
   [junit4]   2> 130181 T368 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:14056 14056
   [junit4]   2> 130255 T368 oas.SolrTestCaseJ4.tearDown ###Ending testOverseerFailure
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=OverseerTest -Dtests.method=testOverseerFailure -Dtests.seed=9F02B25F17FA0C85 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=da -Dtests.timezone=EAT -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   3.26s J0 | OverseerTest.testOverseerFailure <<<
   [junit4]    > Throwable #1: org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode = NodeExists for /collections/collection1/leaders/shard1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([9F02B25F17FA0C85:9B0A3DAC055FE3A4]:0)
   [junit4]    > 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:119)
   [junit4]    > 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]    > 	at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
   [junit4]    > 	at org.apache.solr.common.cloud.SolrZkClient$10.execute(SolrZkClient.java:429)
   [junit4]    > 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]    > 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:426)
   [junit4]    > 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:383)
   [junit4]    > 	at org.apache.solr.common.cloud.SolrZkClient.makePath(SolrZkClient.java:370)
   [junit4]    > 	at org.apache.solr.cloud.ShardLeaderElectionContextBase.runLeaderProcess(ElectionContext.java:112)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.runIamLeaderProcess(LeaderElector.java:164)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:108)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:156)
   [junit4]    > 	at org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:289)
   [junit4]    > 	at org.apache.solr.cloud.OverseerTest$MockZKController.publishState(OverseerTest.java:154)
   [junit4]    > 	at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:590)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:724)
   [junit4]   2> 130330 T368 oas.SolrTestCaseJ4.setUp ###Starting testShardAssignmentBigger
   [junit4]   2> 130330 T368 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 130331 T390 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 130432 T368 oasc.ZkTestServer.run start zk server on port:41651
   [junit4]   2> 130433 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130439 T396 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@57fe2b6 name:ZooKeeperConnection Watcher:127.0.0.1:41651 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130440 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130493 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130501 T398 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2b4985d2 name:ZooKeeperConnection Watcher:127.0.0.1:41651 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130501 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130502 T368 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 130506 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130515 T400 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c325a3b name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130515 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130516 T368 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 130519 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130521 T402 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@45d7d580 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130521 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130523 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 130526 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 130531 T368 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:41651_solr
   [junit4]   2> 130531 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 130534 T368 oasc.Overseer.start Overseer (id=91331710825070595-127.0.0.1:41651_solr-n_0000000000) starting
   [junit4]   2> 130537 T368 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 130541 T368 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 130545 T368 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 130548 T404 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 130549 T368 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 130549 T404 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 130549 T403 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 130553 T368 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 130563 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130566 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130569 T406 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b1451ed name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130569 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130571 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130574 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node0
   [junit4]   2> 130577 T400 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 130577 T406 oascc.ZkStateReader$3.process Updating live nodes... (1)
   [junit4]   2> 130578 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130580 T408 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@a71217a name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130580 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130582 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130585 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node1
   [junit4]   2> 130588 T408 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 130588 T406 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 130588 T400 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 130589 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130591 T410 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1209739c name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130591 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130593 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130596 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node2
   [junit4]   2> 130599 T406 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 130600 T408 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 130599 T410 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 130599 T400 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 130600 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130603 T412 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@183b5819 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130603 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130605 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130607 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node3
   [junit4]   2> 130611 T406 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 130611 T410 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 130611 T400 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 130611 T412 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 130611 T408 oascc.ZkStateReader$3.process Updating live nodes... (4)
   [junit4]   2> 130612 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130614 T414 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3bd33b15 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130614 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130616 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130619 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node4
   [junit4]   2> 130622 T410 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 130622 T406 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 130622 T408 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 130622 T400 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 130622 T412 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 130622 T414 oascc.ZkStateReader$3.process Updating live nodes... (5)
   [junit4]   2> 130623 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130625 T416 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c40a6f8 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130625 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130627 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130630 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node5
   [junit4]   2> 130633 T400 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 130633 T412 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 130633 T408 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 130633 T410 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 130633 T406 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 130634 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130634 T414 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 130634 T416 oascc.ZkStateReader$3.process Updating live nodes... (6)
   [junit4]   2> 130637 T418 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1db9b30c name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130637 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130639 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130641 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node6
   [junit4]   2> 130645 T414 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 130645 T418 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 130645 T406 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 130646 T400 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 130645 T416 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 130645 T408 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 130645 T412 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 130645 T410 oascc.ZkStateReader$3.process Updating live nodes... (7)
   [junit4]   2> 130646 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130648 T420 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@61a05acb name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130649 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130650 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130653 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node7
   [junit4]   2> 130657 T400 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 130657 T410 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 130657 T416 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 130658 T408 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 130657 T414 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 130657 T418 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 130657 T406 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 130657 T412 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 130657 T420 oascc.ZkStateReader$3.process Updating live nodes... (8)
   [junit4]   2> 130658 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130660 T422 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3bf7a927 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130661 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130662 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130665 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node8
   [junit4]   2> 130669 T408 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 130669 T412 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 130669 T400 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 130670 T422 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 130669 T410 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 130669 T414 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 130669 T406 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 130669 T418 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 130669 T416 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 130670 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130670 T420 oascc.ZkStateReader$3.process Updating live nodes... (9)
   [junit4]   2> 130673 T424 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1ccefbbd name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130673 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130675 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130678 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node9
   [junit4]   2> 130681 T416 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 130682 T412 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 130682 T424 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 130682 T422 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 130682 T406 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 130682 T414 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 130681 T408 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 130681 T418 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 130683 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130682 T420 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 130682 T400 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 130682 T410 oascc.ZkStateReader$3.process Updating live nodes... (10)
   [junit4]   2> 130685 T426 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6466fd34 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130686 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130688 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130690 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node10
   [junit4]   2> 130694 T418 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 130694 T416 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 130694 T424 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 130695 T410 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 130695 T412 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 130694 T408 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 130695 T400 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 130695 T420 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 130695 T422 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 130694 T414 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 130694 T426 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 130694 T406 oascc.ZkStateReader$3.process Updating live nodes... (11)
   [junit4]   2> 130696 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130698 T428 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3182afc9 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130699 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130700 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130703 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node11
   [junit4]   2> 130707 T422 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130707 T424 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130708 T410 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130707 T420 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130708 T412 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130707 T428 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130707 T414 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130707 T400 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130709 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130708 T418 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130708 T416 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130708 T406 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130708 T408 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130708 T426 oascc.ZkStateReader$3.process Updating live nodes... (12)
   [junit4]   2> 130711 T430 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2c2fe4e7 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130712 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130714 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130716 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node12
   [junit4]   2> 130720 T424 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130720 T430 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130721 T412 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130721 T400 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130721 T422 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130720 T414 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130721 T418 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130720 T408 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130720 T406 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130720 T416 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130722 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130721 T410 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130721 T426 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130721 T428 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130721 T420 oascc.ZkStateReader$3.process Updating live nodes... (13)
   [junit4]   2> 130724 T432 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2fecb886 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130725 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130727 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130729 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node13
   [junit4]   2> 130733 T426 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130733 T412 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130734 T424 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130734 T418 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130734 T428 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130734 T410 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130734 T432 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130733 T420 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130735 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130735 T430 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130735 T406 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130734 T416 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130734 T414 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130734 T422 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130734 T408 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130734 T400 oascc.ZkStateReader$3.process Updating live nodes... (14)
   [junit4]   2> 130737 T434 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1672f0a6 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130739 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130740 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130743 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node14
   [junit4]   2> 130747 T406 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130747 T426 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130748 T420 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130748 T410 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130748 T432 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130748 T414 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130748 T408 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130747 T424 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130747 T434 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130749 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130749 T428 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130749 T400 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130749 T422 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130748 T412 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130748 T430 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130748 T418 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130748 T416 oascc.ZkStateReader$3.process Updating live nodes... (15)
   [junit4]   2> 130751 T436 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@765746ca name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130753 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130754 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130757 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node15
   [junit4]   2> 130761 T414 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130761 T422 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130762 T430 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130762 T406 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130762 T420 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130762 T434 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130762 T416 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130761 T428 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130761 T400 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130761 T424 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130761 T408 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130761 T436 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130763 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130763 T432 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130762 T410 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130762 T412 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130762 T418 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130762 T426 oascc.ZkStateReader$3.process Updating live nodes... (16)
   [junit4]   2> 130765 T438 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64862576 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130767 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130768 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130771 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node16
   [junit4]   2> 130775 T432 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130776 T434 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130776 T422 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130776 T430 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130776 T424 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130777 T438 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130776 T418 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130776 T420 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130776 T410 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130776 T412 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130776 T406 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130776 T426 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130778 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130777 T408 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130780 T440 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@12892419 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130777 T414 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130777 T428 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130777 T436 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130777 T416 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130777 T400 oascc.ZkStateReader$3.process Updating live nodes... (17)
   [junit4]   2> 130781 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130784 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130787 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node17
   [junit4]   2> 130791 T406 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130791 T434 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130791 T418 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130792 T436 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130792 T420 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130792 T400 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130792 T424 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130793 T408 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130792 T428 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130792 T412 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130791 T410 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130791 T426 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130791 T432 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130791 T416 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130791 T440 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130795 T442 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@68e6e00 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130793 T422 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130793 T430 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130792 T414 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130792 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130792 T438 oascc.ZkStateReader$3.process Updating live nodes... (18)
   [junit4]   2> 130797 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130799 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130802 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node18
   [junit4]   2> 130806 T422 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130807 T436 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130807 T414 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130808 T420 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130808 T426 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130807 T438 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130807 T428 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130807 T430 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130807 T424 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130807 T400 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130809 T432 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130809 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130809 T440 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130808 T418 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130808 T406 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130808 T434 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130808 T416 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130808 T412 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130808 T410 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130808 T442 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130807 T408 oascc.ZkStateReader$3.process Updating live nodes... (19)
   [junit4]   2> 130811 T444 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@97d9475 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130813 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130815 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130818 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node19
   [junit4]   2> 130822 T420 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130823 T426 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130823 T440 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130823 T434 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130824 T418 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130824 T438 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130824 T430 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130823 T406 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130823 T412 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130823 T442 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130823 T410 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130823 T432 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130823 T416 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130825 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130825 T444 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130825 T436 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130824 T408 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130824 T414 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130824 T424 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130824 T400 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130824 T422 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130824 T428 oascc.ZkStateReader$3.process Updating live nodes... (20)
   [junit4]   2> 130828 T446 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3076f54c name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130831 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130834 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130837 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node20
   [junit4]   2> 130842 T428 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130843 T444 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130843 T432 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130843 T412 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130843 T430 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130843 T438 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130844 T406 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130842 T424 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130842 T400 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130845 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130844 T446 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130844 T434 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130844 T410 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130844 T416 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130844 T420 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130844 T414 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130844 T418 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130844 T442 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130843 T440 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130843 T436 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130843 T408 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130843 T422 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130843 T426 oascc.ZkStateReader$3.process Updating live nodes... (21)
   [junit4]   2> 130848 T448 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4a40ce0a name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130850 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130852 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130855 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node21
   [junit4]   2> 130859 T446 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130860 T410 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130860 T420 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130860 T416 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130860 T440 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130860 T418 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130859 T406 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130859 T448 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130859 T434 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130862 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130862 T428 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130861 T430 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130861 T400 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130861 T424 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130861 T422 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130861 T436 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130861 T408 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130861 T438 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130861 T444 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130861 T414 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130861 T432 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130860 T412 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130860 T442 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130860 T426 oascc.ZkStateReader$3.process Updating live nodes... (22)
   [junit4]   2> 130865 T450 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c4b88a5 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130868 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130869 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130872 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node22
   [junit4]   2> 130877 T406 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130877 T416 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130878 T408 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130878 T422 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130878 T432 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130879 T444 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130879 T420 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130877 T446 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130879 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130877 T426 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130877 T448 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130877 T450 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130877 T434 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130877 T418 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130879 T410 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130882 T452 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@16d11a0e name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130879 T412 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130879 T430 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130878 T440 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130878 T436 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130878 T414 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130878 T442 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130878 T400 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130878 T428 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130878 T424 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130878 T438 oascc.ZkStateReader$3.process Updating live nodes... (23)
   [junit4]   2> 130882 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130887 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130890 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node23
   [junit4]   2> 130909 T408 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130909 T424 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130910 T452 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130910 T428 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130911 T400 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130911 T412 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130911 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130910 T430 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130910 T450 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130910 T436 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130910 T414 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130910 T442 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130910 T406 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130914 T454 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2e709107 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130910 T434 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130910 T418 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130910 T422 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130909 T416 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130909 T426 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130909 T444 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130909 T432 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130909 T438 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130914 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130911 T446 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130911 T448 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130911 T420 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130911 T410 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130911 T440 oascc.ZkStateReader$3.process Updating live nodes... (24)
   [junit4]   2> 130918 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130921 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node24
   [junit4]   2> 130926 T408 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130926 T426 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130926 T406 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130927 T434 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130927 T420 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130927 T440 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130927 T452 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130927 T400 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130927 T430 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130927 T428 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130927 T422 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130926 T418 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130926 T416 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130926 T432 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130926 T438 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130926 T444 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130926 T424 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130929 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130928 T410 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130928 T414 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130928 T454 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130928 T442 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130928 T446 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130928 T448 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130928 T412 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130927 T436 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130927 T450 oascc.ZkStateReader$3.process Updating live nodes... (25)
   [junit4]   2> 130931 T456 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@72e0e90a name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130935 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130936 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130939 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node25
   [junit4]   2> 130943 T410 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130944 T412 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130943 T420 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130944 T422 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130945 T424 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130945 T434 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130945 T414 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130946 T452 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130945 T432 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130945 T440 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130945 T444 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130945 T408 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130945 T406 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130944 T430 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130944 T428 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130944 T400 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130944 T456 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130944 T442 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130944 T446 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130944 T448 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130944 T450 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130949 T458 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5cd2aec1 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130947 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130946 T418 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130946 T426 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130946 T436 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130946 T416 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130945 T454 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130945 T438 oascc.ZkStateReader$3.process Updating live nodes... (26)
   [junit4]   2> 130951 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130954 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130957 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node26
   [junit4]   2> 130961 T408 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130962 T424 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130962 T458 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130962 T420 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130961 T426 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130963 T428 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130961 T444 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130964 T452 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130964 T430 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130964 T422 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130963 T450 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130963 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130963 T446 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130963 T416 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130963 T432 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130963 T438 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130963 T448 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130963 T410 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130962 T440 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130962 T456 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130962 T412 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130962 T442 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130962 T414 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130962 T406 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130962 T400 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130962 T418 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130962 T434 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130966 T460 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@4e09645c name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130964 T454 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130964 T436 oascc.ZkStateReader$3.process Updating live nodes... (27)
   [junit4]   2> 130970 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130972 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130975 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node27
   [junit4]   2> 130980 T416 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130980 T438 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130980 T424 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130981 T442 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130981 T444 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130980 T418 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130982 T440 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130982 T436 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130982 T448 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130983 T400 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130980 T406 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130983 T368 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 130980 T432 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130982 T414 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130982 T422 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130982 T454 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130982 T452 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130982 T446 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130986 T462 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@42ff95b0 name:ZooKeeperConnection Watcher:127.0.0.1:41651/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 130982 T458 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130982 T430 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130981 T420 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130981 T412 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130981 T428 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130981 T410 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130981 T450 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130981 T456 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130981 T460 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130981 T408 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130980 T434 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130980 T426 oascc.ZkStateReader$3.process Updating live nodes... (28)
   [junit4]   2> 130986 T368 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 130991 T368 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 130994 T368 oascc.SolrZkClient.makePath makePath: /live_nodes/node28
   [junit4]   2> 130998 T460 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 130998 T440 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 130998 T412 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 130998 T448 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 131000 T430 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 131000 T422 oascc.ZkStateReader$3.process Updating live nodes... (29)
   [junit4]   2> 131000 T434 oascc.ZkState

[...truncated too long message...]

nit4]   2> 152204 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 152208 T368 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:26420_solr
   [junit4]   2> 152208 T368 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 152210 T368 oasc.Overseer.start Overseer (id=91331712246415364-127.0.0.1:26420_solr-n_0000000000) starting
   [junit4]   2> 152212 T368 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 152215 T368 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 152218 T368 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 152220 T898 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 152220 T898 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 152220 T897 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 152222 T897 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 152223 T897 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=12 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"recovering",
   [junit4]   2> 	  "node_name":"node1",
   [junit4]   2> 	  "core":"core1",
   [junit4]   2> 	  "core_node_name":"node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"12",
   [junit4]   2> 	  "base_url":"http://node1/solr/"}
   [junit4]   2> 152223 T897 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2, shard3, shard4, shard5, shard6, shard7, shard8, shard9, shard10, shard11, shard12]
   [junit4]   2> 152224 T897 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard10
   [junit4]   2> 152226 T896 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 152227 T894 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 152227 T888 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 152723 T368 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard10/election
   [junit4]   2> 152753 T368 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard10
   [junit4]   2> 152766 T896 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 152766 T368 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 152767 T897 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 152775 T888 oascc.ZkStateReader$3.process Updating live nodes... (0)
   [junit4]   2> 152775 T894 oascc.ZkStateReader$3.process Updating live nodes... (0)
   [junit4]   2> 152778 T368 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:26420 26420
   [junit4]   2> 152843 T368 oas.SolrTestCaseJ4.tearDown ###Ending testPlaceholders
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 152846 T368 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> 152846 T368 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 153322 T881 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 153322 T881 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 153323 T881 oasc.LeaderElector$1.process WARN  org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/leader_elect/shard3/election
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1468)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:257)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$6.execute(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:254)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:92)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector.access$000(LeaderElector.java:55)
   [junit4]   2> 	at org.apache.solr.cloud.LeaderElector$1.process(LeaderElector.java:137)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
   [junit4]   2> 
   [junit4]   2> 155847 T368 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Lucene46: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=crazy): {}, locale=da, timezone=EAT
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=16,threads=1,free=75672752,total=145752064
   [junit4]   2> NOTE: All tests run in this JVM: [AlternateDirectoryTest, TestRemoteStreaming, TestSystemIdResolver, TestCloudManagedSchemaAddField, TestJmxMonitoredMap, TestValueSourceCache, CircularListTest, ClusterStateUpdateTest, TestCollationField, QueryElevationComponentTest, SuggestComponentTest, RequestHandlersTest, TestSchemaResource, OverseerTest]
   [junit4] Completed on J0 in 28.84s, 8 tests, 1 error <<< FAILURES!

[...truncated 987 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:471: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:451: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:490: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1278: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:911: There were test failures: 363 suites, 1570 tests, 1 error, 36 ignored (8 assumptions)

Total time: 63 minutes 58 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1913 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1913/

All tests passed

Build Log:
[...truncated 28417 lines...]
check-licenses:
     [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jcl-over-slf4j-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jcl-over-slf4j-1.6.6.jar.sha1

 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jul-to-slf4j-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jul-to-slf4j-1.6.6.jar.sha1
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/slf4j-api-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/slf4j-api-1.6.6.jar.sha1

[...truncated 3 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:471: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:64: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.

Total time: 108 minutes 19 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1912 - Still Failing

Posted by Robert Muir <rc...@gmail.com>.
I committed a fix.

On Wed, Feb 26, 2014 at 6:01 PM, Apache Jenkins Server
<je...@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1912/
>
> All tests passed
>
> Build Log:
> [...truncated 28267 lines...]
> check-licenses:
>      [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
>  [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jcl-over-slf4j-1.6.6.jar
>  [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jcl-over-slf4j-1.6.6.jar.sha1
>
>  [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jul-to-slf4j-1.6.6.jar
>  [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jul-to-slf4j-1.6.6.jar.sha1
>  [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/slf4j-api-1.6.6.jar
>  [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/slf4j-api-1.6.6.jar.sha1
>
> [...truncated 3 lines...]
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:471: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:64: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.
>
> Total time: 107 minutes 7 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1912 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1912/

All tests passed

Build Log:
[...truncated 28267 lines...]
check-licenses:
     [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jcl-over-slf4j-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jcl-over-slf4j-1.6.6.jar.sha1

 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/jul-to-slf4j-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/jul-to-slf4j-1.6.6.jar.sha1
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/slf4j-api-1.6.6.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/slf4j-api-1.6.6.jar.sha1

[...truncated 3 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:471: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:64: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.

Total time: 107 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1911 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1911/

3 tests failed.
REGRESSION:  org.apache.solr.cloud.ChaosMonkeySafeLeaderTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([4D75887B35EB759B]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeySafeLeaderTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([4D75887B35EB759B]:0)


REGRESSION:  org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest.testDistribSearch

Error Message:
this codec cannot write docvalues

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: this codec cannot write docvalues
	at __randomizedtesting.SeedInfo.seed([4D75887B35EB759B:CC93066342B415A7]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:497)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
	at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:168)
	at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:146)
	at org.apache.solr.BaseDistributedSearchTestCase.commit(BaseDistributedSearchTestCase.java:505)
	at org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest.doTest(DistributedQueryComponentOptimizationTest.java:66)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:724)




Build Log:
[...truncated 10577 lines...]
   [junit4] Suite: org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest
   [junit4]   2> 4988487 T16025 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 4988488 T16025 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /v_fzy/e
   [junit4]   2> 4988491 T16025 oas.SolrTestCaseJ4.initCore ####initCore
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solrtest-DistributedQueryComponentOptimizationTest-1393371199395
   [junit4]   2> 4988492 T16025 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 4988493 T16025 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 4988494 T16025 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 4988495 T16025 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 4988643 T16025 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 4988741 T16025 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4988743 T16025 oass.IndexSchema.readSchema Reading Solr Schema from schema-custom-field.xml
   [junit4]   2> 4988750 T16025 oass.IndexSchema.readSchema [null] Schema name=test-custom-field-sort
   [junit4]   2> 4988793 T16025 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 4988797 T16025 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 4988799 T16025 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4988799 T16025 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr
   [junit4]   2> 4988800 T16025 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/'
   [junit4]   2> 4988894 T16025 oasc.CoreContainer.<init> New CoreContainer 1514486332
   [junit4]   2> 4988895 T16025 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/]
   [junit4]   2> 4988897 T16025 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 4988897 T16025 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 4988898 T16025 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 4988898 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 4988899 T16025 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 4988899 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 4988900 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 4988900 T16025 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 4988901 T16025 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 4988917 T16025 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4988917 T16025 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4988917 T16025 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 4988924 T16026 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1
   [junit4]   2> 4988925 T16026 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 4988926 T16026 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 4988927 T16026 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 4988927 T16026 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 4989010 T16026 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 4989107 T16026 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4989108 T16026 oass.IndexSchema.readSchema Reading Solr Schema from schema-custom-field.xml
   [junit4]   2> 4989115 T16026 oass.IndexSchema.readSchema [collection1] Schema name=test-custom-field-sort
   [junit4]   2> 4989155 T16026 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 4989158 T16026 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 4989160 T16026 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 4989161 T16026 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solrtest-DistributedQueryComponentOptimizationTest-1393371199395/
   [junit4]   2> 4989161 T16026 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5738ff7c
   [junit4]   2> 4989162 T16026 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solrtest-DistributedQueryComponentOptimizationTest-1393371199395
   [junit4]   2> 4989163 T16026 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solrtest-DistributedQueryComponentOptimizationTest-1393371199395/index/
   [junit4]   2> 4989163 T16026 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solrtest-DistributedQueryComponentOptimizationTest-1393371199395/index' doesn't exist. Creating new index...
   [junit4]   2> 4989164 T16026 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solrtest-DistributedQueryComponentOptimizationTest-1393371199395/index
   [junit4]   2> 4989164 T16026 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=27, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.24571304246398445]
   [junit4]   2> 4989165 T16026 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@37fa5b99 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d6d88ea)),segFN=segments_1,generation=1}
   [junit4]   2> 4989166 T16026 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4989167 T16026 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 4989171 T16026 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4989172 T16026 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4989172 T16026 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 4989173 T16026 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 4989173 T16026 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4989173 T16026 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4989174 T16026 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 4989174 T16026 oasup.UniqFieldsUpdateProcessorFactory.init WARN Use of the 'fields' init param in UniqFieldsUpdateProcessorFactory is deprecated, please use 'fieldName' (or another FieldMutatingUpdateProcessorFactory selector option) instead
   [junit4]   2> 4989175 T16026 oasup.UniqFieldsUpdateProcessorFactory.init Replacing 'fields' init param with (individual) 'fieldName' params
   [junit4]   2> 4989175 T16026 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 4989176 T16026 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4989176 T16026 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4989177 T16026 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4989177 T16026 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4989178 T16026 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 4989178 T16026 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 4989179 T16026 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 4989180 T16026 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 4989180 T16026 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 4989181 T16026 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 4989182 T16026 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 4989182 T16026 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 4989183 T16026 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 4989183 T16026 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 4989184 T16026 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 4989184 T16026 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 4989185 T16026 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989186 T16026 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989186 T16026 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989187 T16026 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989188 T16026 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989188 T16026 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989189 T16026 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989189 T16026 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989190 T16026 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 4989191 T16026 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 4989208 T16026 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4989210 T16026 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 4989214 T16026 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 4989214 T16026 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 4989215 T16026 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=55.7451171875, floorSegmentMB=0.6640625, forceMergeDeletesPctAllowed=29.335149011310897, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 4989216 T16026 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@37fa5b99 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2d6d88ea)),segFN=segments_1,generation=1}
   [junit4]   2> 4989216 T16026 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4989217 T16026 oass.SolrIndexSearcher.<init> Opening Searcher@32b06c9b[collection1] main
   [junit4]   2> 4989217 T16026 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 4989225 T16026 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 4989306 T16027 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@32b06c9b[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 4989313 T16026 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 4989330 T16025 oas.SolrTestCaseJ4.initCore ####initCore end
   [junit4]   2> 4989338 T16025 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 4989342 T16025 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 4989347 T16025 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 4989351 T16025 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:63664
   [junit4]   2> 4989354 T16025 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 4989354 T16025 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4989355 T16025 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr
   [junit4]   2> 4989356 T16025 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/'
   [junit4]   2> 4989404 T16025 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/solr.xml
   [junit4]   2> 4989562 T16025 oasc.CoreContainer.<init> New CoreContainer 1766085693
   [junit4]   2> 4989564 T16025 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/]
   [junit4]   2> 4989566 T16025 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 4989566 T16025 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: https
   [junit4]   2> 4989567 T16025 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 4989568 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 4989568 T16025 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 4989569 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 4989569 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 4989570 T16025 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 4989571 T16025 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 4989586 T16025 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4989587 T16025 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4989588 T16025 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 4989594 T16038 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1
   [junit4]   2> 4989595 T16038 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 4989596 T16038 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 4989597 T16038 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 4989598 T16038 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 4989688 T16038 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 4989785 T16038 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4989786 T16038 oass.IndexSchema.readSchema Reading Solr Schema from schema-custom-field.xml
   [junit4]   2> 4989794 T16038 oass.IndexSchema.readSchema [collection1] Schema name=test-custom-field-sort
   [junit4]   2> 4989836 T16038 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 4989840 T16038 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 4989841 T16038 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 4989842 T16038 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/, dataDir=./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/control/data/
   [junit4]   2> 4989842 T16038 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5738ff7c
   [junit4]   2> 4989844 T16038 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/control/data
   [junit4]   2> 4989844 T16038 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/control/data/index/
   [junit4]   2> 4989844 T16038 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 4989845 T16038 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/control/data/index
   [junit4]   2> 4989846 T16038 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=27, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.24571304246398445]
   [junit4]   2> 4989847 T16038 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1f10229f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2311d9c2)),segFN=segments_1,generation=1}
   [junit4]   2> 4989847 T16038 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4989848 T16038 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 4989853 T16038 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4989854 T16038 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4989854 T16038 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 4989855 T16038 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 4989855 T16038 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4989856 T16038 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4989856 T16038 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 4989857 T16038 oasup.UniqFieldsUpdateProcessorFactory.init WARN Use of the 'fields' init param in UniqFieldsUpdateProcessorFactory is deprecated, please use 'fieldName' (or another FieldMutatingUpdateProcessorFactory selector option) instead
   [junit4]   2> 4989857 T16038 oasup.UniqFieldsUpdateProcessorFactory.init Replacing 'fields' init param with (individual) 'fieldName' params
   [junit4]   2> 4989857 T16038 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 4989858 T16038 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4989859 T16038 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4989859 T16038 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4989859 T16038 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4989860 T16038 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 4989860 T16038 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 4989861 T16038 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 4989862 T16038 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 4989863 T16038 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 4989864 T16038 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 4989865 T16038 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 4989866 T16038 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 4989867 T16038 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 4989867 T16038 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 4989867 T16038 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 4989868 T16038 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 4989869 T16038 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989870 T16038 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989871 T16038 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989871 T16038 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989872 T16038 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989873 T16038 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989874 T16038 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989875 T16038 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4989875 T16038 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 4989876 T16038 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 4989879 T16038 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4989881 T16038 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 4989887 T16038 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 4989887 T16038 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 4989888 T16038 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=55.7451171875, floorSegmentMB=0.6640625, forceMergeDeletesPctAllowed=29.335149011310897, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 4989889 T16038 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@1f10229f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2311d9c2)),segFN=segments_1,generation=1}
   [junit4]   2> 4989890 T16038 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4989890 T16038 oass.SolrIndexSearcher.<init> Opening Searcher@3f2be016[collection1] main
   [junit4]   2> 4989891 T16038 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 4989926 T16038 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 4989987 T16039 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3f2be016[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 4989996 T16038 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 4989996 T16025 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 4989997 T16025 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 4990002 T16025 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 4990007 T16025 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 4990010 T16025 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:47131
   [junit4]   2> 4990014 T16025 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 4990014 T16025 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4990015 T16025 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr
   [junit4]   2> 4990015 T16025 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/'
   [junit4]   2> 4990064 T16025 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/solr.xml
   [junit4]   2> 4990163 T16025 oasc.CoreContainer.<init> New CoreContainer 1449787431
   [junit4]   2> 4990165 T16025 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/]
   [junit4]   2> 4990167 T16025 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 4990167 T16025 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: https
   [junit4]   2> 4990168 T16025 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 4990169 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 4990169 T16025 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 4990170 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 4990171 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 4990171 T16025 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 4990172 T16025 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 4990188 T16025 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4990189 T16025 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4990189 T16025 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 4990196 T16050 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1
   [junit4]   2> 4990196 T16050 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 4990198 T16050 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 4990199 T16050 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 4990199 T16050 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 4990289 T16050 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 4990386 T16050 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4990387 T16050 oass.IndexSchema.readSchema Reading Solr Schema from schema-custom-field.xml
   [junit4]   2> 4990394 T16050 oass.IndexSchema.readSchema [collection1] Schema name=test-custom-field-sort
   [junit4]   2> 4990436 T16050 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 4990439 T16050 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 4990441 T16050 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 4990442 T16050 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/, dataDir=./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard0/data/
   [junit4]   2> 4990442 T16050 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5738ff7c
   [junit4]   2> 4990443 T16050 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard0/data
   [junit4]   2> 4990444 T16050 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard0/data/index/
   [junit4]   2> 4990444 T16050 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard0/data/index' doesn't exist. Creating new index...
   [junit4]   2> 4990445 T16050 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard0/data/index
   [junit4]   2> 4990445 T16050 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=27, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.24571304246398445]
   [junit4]   2> 4990447 T16050 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@3ab4eb86 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1abf9b2a)),segFN=segments_1,generation=1}
   [junit4]   2> 4990447 T16050 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4990448 T16050 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 4990453 T16050 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4990454 T16050 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4990454 T16050 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 4990455 T16050 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 4990455 T16050 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4990455 T16050 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4990456 T16050 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 4990456 T16050 oasup.UniqFieldsUpdateProcessorFactory.init WARN Use of the 'fields' init param in UniqFieldsUpdateProcessorFactory is deprecated, please use 'fieldName' (or another FieldMutatingUpdateProcessorFactory selector option) instead
   [junit4]   2> 4990457 T16050 oasup.UniqFieldsUpdateProcessorFactory.init Replacing 'fields' init param with (individual) 'fieldName' params
   [junit4]   2> 4990457 T16050 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 4990457 T16050 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4990458 T16050 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4990459 T16050 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4990459 T16050 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4990459 T16050 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 4990460 T16050 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 4990461 T16050 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 4990462 T16050 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 4990463 T16050 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 4990464 T16050 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 4990464 T16050 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 4990465 T16050 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 4990466 T16050 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 4990466 T16050 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 4990467 T16050 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 4990468 T16050 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 4990468 T16050 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4990469 T16050 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4990470 T16050 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4990471 T16050 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4990472 T16050 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4990472 T16050 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4990473 T16050 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4990474 T16050 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4990475 T16050 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 4990476 T16050 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 4990493 T16050 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4990495 T16050 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 4990501 T16050 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 4990501 T16050 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 4990502 T16050 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=55.7451171875, floorSegmentMB=0.6640625, forceMergeDeletesPctAllowed=29.335149011310897, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 4990503 T16050 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@3ab4eb86 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1abf9b2a)),segFN=segments_1,generation=1}
   [junit4]   2> 4990503 T16050 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4990504 T16050 oass.SolrIndexSearcher.<init> Opening Searcher@51f0468a[collection1] main
   [junit4]   2> 4990504 T16050 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 4990541 T16050 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 4990601 T16051 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@51f0468a[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 4990610 T16050 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 4990610 T16025 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 4990611 T16025 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 4990616 T16025 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 4990621 T16025 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 4990624 T16025 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:62277
   [junit4]   2> 4990627 T16025 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 4990628 T16025 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4990628 T16025 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr
   [junit4]   2> 4990629 T16025 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/'
   [junit4]   2> 4990677 T16025 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/solr.xml
   [junit4]   2> 4990777 T16025 oasc.CoreContainer.<init> New CoreContainer 1296771769
   [junit4]   2> 4990778 T16025 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/]
   [junit4]   2> 4990780 T16025 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 4990781 T16025 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: https
   [junit4]   2> 4990782 T16025 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 4990782 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 4990783 T16025 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 4990783 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 4990784 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 4990785 T16025 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 4990785 T16025 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 4990801 T16025 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4990802 T16025 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4990802 T16025 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 4990809 T16062 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1
   [junit4]   2> 4990809 T16062 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 4990811 T16062 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 4990811 T16062 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 4990812 T16062 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 4990902 T16062 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 4991046 T16062 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4991048 T16062 oass.IndexSchema.readSchema Reading Solr Schema from schema-custom-field.xml
   [junit4]   2> 4991055 T16062 oass.IndexSchema.readSchema [collection1] Schema name=test-custom-field-sort
   [junit4]   2> 4991097 T16062 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 4991100 T16062 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 4991102 T16062 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 4991102 T16062 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/, dataDir=./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard1/data/
   [junit4]   2> 4991103 T16062 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5738ff7c
   [junit4]   2> 4991104 T16062 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard1/data
   [junit4]   2> 4991104 T16062 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard1/data/index/
   [junit4]   2> 4991105 T16062 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 4991105 T16062 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard1/data/index
   [junit4]   2> 4991106 T16062 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=27, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.24571304246398445]
   [junit4]   2> 4991107 T16062 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@50a3e688 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f86d67)),segFN=segments_1,generation=1}
   [junit4]   2> 4991108 T16062 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4991108 T16062 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 4991114 T16062 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4991114 T16062 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4991115 T16062 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 4991115 T16062 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 4991116 T16062 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4991116 T16062 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4991116 T16062 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 4991117 T16062 oasup.UniqFieldsUpdateProcessorFactory.init WARN Use of the 'fields' init param in UniqFieldsUpdateProcessorFactory is deprecated, please use 'fieldName' (or another FieldMutatingUpdateProcessorFactory selector option) instead
   [junit4]   2> 4991117 T16062 oasup.UniqFieldsUpdateProcessorFactory.init Replacing 'fields' init param with (individual) 'fieldName' params
   [junit4]   2> 4991118 T16062 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 4991118 T16062 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4991119 T16062 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4991119 T16062 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4991120 T16062 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4991120 T16062 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 4991121 T16062 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 4991122 T16062 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 4991122 T16062 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 4991123 T16062 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 4991124 T16062 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 4991125 T16062 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 4991126 T16062 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 4991127 T16062 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 4991127 T16062 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 4991127 T16062 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 4991128 T16062 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 4991129 T16062 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991130 T16062 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991131 T16062 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991131 T16062 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991132 T16062 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991133 T16062 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991134 T16062 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991134 T16062 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991135 T16062 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 4991136 T16062 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 4991153 T16062 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4991155 T16062 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 4991161 T16062 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 4991161 T16062 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 4991162 T16062 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=55.7451171875, floorSegmentMB=0.6640625, forceMergeDeletesPctAllowed=29.335149011310897, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 4991163 T16062 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@50a3e688 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1f86d67)),segFN=segments_1,generation=1}
   [junit4]   2> 4991164 T16062 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4991164 T16062 oass.SolrIndexSearcher.<init> Opening Searcher@58f4316a[collection1] main
   [junit4]   2> 4991165 T16062 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 4991198 T16062 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 4991259 T16063 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@58f4316a[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 4991267 T16062 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 4991268 T16025 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 4991269 T16025 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 4991274 T16025 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 4991278 T16025 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 4991282 T16025 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:54201
   [junit4]   2> 4991285 T16025 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 4991285 T16025 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 4991286 T16025 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr
   [junit4]   2> 4991287 T16025 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/'
   [junit4]   2> 4991336 T16025 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/solr.xml
   [junit4]   2> 4991435 T16025 oasc.CoreContainer.<init> New CoreContainer 352063868
   [junit4]   2> 4991437 T16025 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/]
   [junit4]   2> 4991439 T16025 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 4991439 T16025 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: https
   [junit4]   2> 4991440 T16025 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 4991441 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 4991441 T16025 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 4991442 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 4991442 T16025 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 4991443 T16025 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 4991444 T16025 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 4991458 T16025 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 4991458 T16025 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 4991459 T16025 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 4991465 T16074 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1
   [junit4]   2> 4991466 T16074 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/'
   [junit4]   2> 4991467 T16074 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/classes/' to classloader
   [junit4]   2> 4991468 T16074 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/.svn/' to classloader
   [junit4]   2> 4991469 T16074 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/lib/README' to classloader
   [junit4]   2> 4991558 T16074 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 4991655 T16074 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 4991656 T16074 oass.IndexSchema.readSchema Reading Solr Schema from schema-custom-field.xml
   [junit4]   2> 4991663 T16074 oass.IndexSchema.readSchema [collection1] Schema name=test-custom-field-sort
   [junit4]   2> 4991705 T16074 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 4991709 T16074 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 4991710 T16074 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 4991711 T16074 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/, dataDir=./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard2/data/
   [junit4]   2> 4991712 T16074 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@5738ff7c
   [junit4]   2> 4991713 T16074 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard2/data
   [junit4]   2> 4991713 T16074 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard2/data/index/
   [junit4]   2> 4991714 T16074 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard2/data/index' doesn't exist. Creating new index...
   [junit4]   2> 4991714 T16074 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.handler.component.DistributedQueryComponentOptimizationTest-1393371200243/shard2/data/index
   [junit4]   2> 4991715 T16074 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=27, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.24571304246398445]
   [junit4]   2> 4991716 T16074 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@39d641b3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3d6aa46d)),segFN=segments_1,generation=1}
   [junit4]   2> 4991717 T16074 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4991717 T16074 oasc.SolrCore.initWriters created xml: solr.XMLResponseWriter
   [junit4]   2> 4991722 T16074 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 4991723 T16074 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 4991724 T16074 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 4991724 T16074 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe-allfields"
   [junit4]   2> 4991724 T16074 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4991725 T16074 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 4991725 T16074 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 4991726 T16074 oasup.UniqFieldsUpdateProcessorFactory.init WARN Use of the 'fields' init param in UniqFieldsUpdateProcessorFactory is deprecated, please use 'fieldName' (or another FieldMutatingUpdateProcessorFactory selector option) instead
   [junit4]   2> 4991726 T16074 oasup.UniqFieldsUpdateProcessorFactory.init Replacing 'fields' init param with (individual) 'fieldName' params
   [junit4]   2> 4991727 T16074 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "uniq-fields"
   [junit4]   2> 4991727 T16074 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 4991728 T16074 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4991728 T16074 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 4991729 T16074 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 4991729 T16074 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 4991729 T16074 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 4991730 T16074 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 4991731 T16074 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 4991732 T16074 oasc.RequestHandlers.initHandlersFromConfig created dismax: solr.SearchHandler
   [junit4]   2> 4991733 T16074 oasc.RequestHandlers.initHandlersFromConfig created mock: org.apache.solr.core.MockQuerySenderListenerReqHandler
   [junit4]   2> 4991734 T16074 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 4991735 T16074 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 4991735 T16074 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 4991736 T16074 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 4991736 T16074 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 4991737 T16074 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 4991738 T16074 oasc.RequestHandlers.initHandlersFromConfig created /terms: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991739 T16074 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991739 T16074 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991740 T16074 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991741 T16074 oasc.RequestHandlers.initHandlersFromConfig created spellCheckWithWordbreak_Direct: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991742 T16074 oasc.RequestHandlers.initHandlersFromConfig created spellCheckCompRH1: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991742 T16074 oasc.RequestHandlers.initHandlersFromConfig created mltrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991743 T16074 oasc.RequestHandlers.initHandlersFromConfig created tvrh: org.apache.solr.handler.component.SearchHandler
   [junit4]   2> 4991744 T16074 oasc.RequestHandlers.initHandlersFromConfig created /mlt: solr.MoreLikeThisHandler
   [junit4]   2> 4991745 T16074 oasc.RequestHandlers.initHandlersFromConfig created /debug/dump: solr.DumpRequestHandler
   [junit4]   2> 4991762 T16074 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 4991764 T16074 oasc.SolrCore.initDeprecatedSupport WARN solrconfig.xml uses deprecated <admin/gettableFiles>, Please update your config to use the ShowFileRequestHandler.
   [junit4]   2> 4991770 T16074 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 4991770 T16074 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 4991771 T16074 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=55.7451171875, floorSegmentMB=0.6640625, forceMergeDeletesPctAllowed=29.335149011310897, segmentsPerTier=35.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 4991772 T16074 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=RateLimitedDirectoryWrapper(MockDirectoryWrapper(RAMDirectory@39d641b3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3d6aa46d)),segFN=segments_1,generation=1}
   [junit4]   2> 4991773 T16074 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 4991773 T16074 oass.SolrIndexSearcher.<init> Opening Searcher@43771604[collection1] main
   [junit4]   2> 4991774 T16074 oashc.SpellCheckComponent.inform Initializing spell checkers
   [junit4]   2> 4991782 T16074 oass.DirectSolrSpellChecker.init init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 4991842 T16075 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@43771604[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 4991851 T16074 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 4991852 T16025 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 4991852 T16025 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C2695 name=collection1 org.apache.solr.core.SolrCore@7517a0df
   [junit4]   2> 4991987 T16031 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1461055602441584640)} 0 1
   [junit4]   2> ASYNC  NEW_CORE C2696 name=collection1 org.apache.solr.core.SolrCore@47d5a92f
   [junit4]   2> 4992117 T16043 C2696 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1461055602577899520)} 0 1
   [junit4]   2> ASYNC  NEW_CORE C2697 name=collection1 org.apache.solr.core.SolrCore@70a8d3e8
   [junit4]   2> 4992248 T16055 C2697 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1461055602715262976)} 0 1
   [junit4]   2> ASYNC  NEW_CORE C2698 name=collection1 org.apache.solr.core.SolrCore@71545e64
   [junit4]   2> 4992379 T16067 C2698 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {deleteByQuery=*:* (-1461055602852626432)} 0 1
   [junit4]   2> 4992385 T16032 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[1 (1461055602858917888)]} 0 1
   [junit4]   2> 4992389 T16044 C2696 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[1 (1461055602863112192)]} 0 1
   [junit4]   2> 4992393 T16033 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[2 (1461055602868355072)]} 0 0
   [junit4]   2> 4992397 T16056 C2697 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[2 (1461055602871500800)]} 0 1
   [junit4]   2> 4992400 T16034 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[3 (1461055602875695104)]} 0 0
   [junit4]   2> 4992405 T16068 C2698 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[3 (1461055602879889408)]} 0 1
   [junit4]   2> 4992408 T16035 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[4 (1461055602884083712)]} 0 0
   [junit4]   2> 4992412 T16045 C2696 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[4 (1461055602888278016)]} 0 0
   [junit4]   2> 4992416 T16036 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[5 (1461055602891423744)]} 0 1
   [junit4]   2> 4992419 T16057 C2697 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[5 (1461055602895618048)]} 0 0
   [junit4]   2> 4992423 T16037 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[6 (1461055602899812352)]} 0 0
   [junit4]   2> 4992427 T16069 C2698 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[6 (1461055602904006656)]} 0 0
   [junit4]   2> 4992431 T16093 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[7 (1461055602908200960)]} 0 0
   [junit4]   2> 4992435 T16046 C2696 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[7 (1461055602912395264)]} 0 0
   [junit4]   2> 4992439 T16096 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[8 (1461055602916589568)]} 0 0
   [junit4]   2> 4992443 T16058 C2697 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[8 (1461055602919735296)]} 0 1
   [junit4]   2> 4992446 T16099 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[9 (1461055602923929600)]} 0 0
   [junit4]   2> 4992450 T16070 C2698 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[9 (1461055602928123904)]} 0 0
   [junit4]   2> 4992454 T16102 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[10 (1461055602932318208)]} 0 0
   [junit4]   2> 4992458 T16047 C2696 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[10 (1461055602936512512)]} 0 0
   [junit4]   2> 4992462 T16105 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[11 (1461055602940706816)]} 0 0
   [junit4]   2> 4992466 T16059 C2697 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[11 (1461055602944901120)]} 0 1
   [junit4]   2> 4992470 T16108 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[12 (1461055602948046848)]} 0 1
   [junit4]   2> 4992473 T16071 C2698 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[12 (1461055602952241152)]} 0 0
   [junit4]   2> 4992477 T16111 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[13 (1461055602956435456)]} 0 0
   [junit4]   2> 4992481 T16048 C2696 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&version=2} {add=[13 (1461055602960629760)]} 0 0
   [junit4]   2> 4992485 T16114 C2695 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 4992486 T16114 C2695 oasup.LogUpdateProcessor.finish [collection1] webapp=/v_fzy/e path=/update params={wt=javabin&waitSearcher=true&softCommit=false&commit=true&version=2} {} 0 1
   [junit4]   2> 4992487 T16114 C2695 oasc.SolrException.log ERROR java.lang.UnsupportedOperationException: this codec cannot write docvalues
   [junit4]   2> 		at org.apache.lucene.codecs.lucene3x.Lucene3xCodec$1.fieldsConsumer(Lucene3xCodec.java:73)
   [junit4]   2> 		at org.apache.lucene.index.DocValuesProcessor.flush(DocValuesProcessor.java:77)
   [junit4]   2> 		at org.apache.lucene.index.TwoStoredFieldsConsumers.flush(TwoStoredFieldsConsumers.java:42)
   [junit4]   2> 		at org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:80)
   [junit4]   2> 		at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:465)
   [junit4]   2> 		at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:506)
   [junit4]   2> 		at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:616)
   [junit4]   2> 		at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2864)
   [junit4]   2> 		at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)
   [junit4]   2> 		at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2989)
   [junit4]   2> 		at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:578)
   [junit4]   2> 		at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:95)
   [junit4]   2> 		at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:64)
   [junit4]   2> 		at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1457)
   [junit4]   2> 		at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1434)
   [junit4]   2> 		at org.apache.solr.update.processor.LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:157)
   [junit4]   2> 		at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:69)
   [junit4]   2> 		at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
   [junit4]   2> 		at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
   [junit4]   2> 		at org.apache.solr.core.SolrCore.execute(SolrCore.java:1916)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:780)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:427)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:217)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:134)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2> 		at org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
   [junit4]   2> 		at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
   [junit4]   2> 		at org.eclipse.jetty.server.ssl.SslSocketConnector$SslConnectorEndPoint.run(SslSocketConnector.java:670)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:724)
   [junit4]   2> 	
   [junit4]   2> 4992489 T16114 C2695 oasc.SolrException.log ERROR null:java.lang.UnsupportedOperationException: this codec cannot write docvalues
   [junit4]   2> 		at org.apache.lucene.codecs.lucene3x.Lucene3xCodec$1.fieldsConsumer(Lucene3xCodec.java:73)
   [junit4]   2> 		at org.apache.lucene.index.DocValuesProcessor.flush(DocValuesProcessor.java:77)
   [junit4]   2> 		at org.apache.lucene.index.TwoStoredFieldsConsumers.flush(TwoStoredFieldsConsumers.java:42)
   [junit4]   2> 		at org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:80)
   [junit4]   2> 		at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:465)
   [junit4]   2> 		at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:506)
   [junit4]   2> 		at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:616)
   [junit4]   2> 		at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2864)
   [junit4]   2> 		at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:3022)
   [junit4]   2> 		at org.apache.lucene.ind

[...truncated too long message...]

lang.Thread.run(Thread.java:724)
   [junit4]   2> 	  60) Thread[id=743, name=HashSessionScavenger-17, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
   [junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
   [junit4]   2> 	  61) Thread[id=744, name=qtp781588329-744 Acceptor0 SocketConnector@127.0.0.1:33558, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
   [junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:530)
   [junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:498)
   [junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
   [junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:724)
   [junit4]   2> 	  62) Thread[id=627, name=qtp1527114287-627 Acceptor0 SocketConnector@127.0.0.1:13711, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
   [junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:530)
   [junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:498)
   [junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
   [junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:724)
   [junit4]   2> 	  63) Thread[id=626, name=HashSessionScavenger-11, state=TIMED_WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.util.TimerThread.mainLoop(Timer.java:552)
   [junit4]   2> 	        at java.util.TimerThread.run(Timer.java:505)
   [junit4]   2> 	  64) Thread[id=718, name=searcherExecutor-396-thread-1, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:724)
   [junit4]   2> 	  65) Thread[id=956, name=searcherExecutor-479-thread-1, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:724)
   [junit4]   2> 	  66) Thread[id=637, name=TEST-ChaosMonkeySafeLeaderTest.testDistribSearch-seed#[4D75887B35EB759B]-SendThread(localhost:11714), state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(Native Method)
   [junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueArrayWrapper.java:164)
   [junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSelectorImpl.java:245)
   [junit4]   2> 	        at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:210)
   [junit4]   2> 	        at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
   [junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:102)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:338)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
   [junit4]   2> 	  67) Thread[id=986, name=qtp1698469511-986 Acceptor0 SocketConnector@127.0.0.1:39533, state=RUNNABLE, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at java.net.PlainSocketImpl.socketAccept(Native Method)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:398)
   [junit4]   2> 	        at java.net.ServerSocket.implAccept(ServerSocket.java:530)
   [junit4]   2> 	        at java.net.ServerSocket.accept(ServerSocket.java:498)
   [junit4]   2> 	        at org.eclipse.jetty.server.bio.SocketConnector.accept(SocketConnector.java:117)
   [junit4]   2> 	        at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:938)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:724)
   [junit4]   2> 	  68) Thread[id=719, name=Thread-268, state=WAITING, group=TGRP-ChaosMonkeySafeLeaderTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at java.lang.Object.wait(Object.java:503)
   [junit4]   2> 	        at org.apache.solr.core.CloserThread.run(CoreContainer.java:1018)
   [junit4]   2> NOTE: test params are: codec=Lucene40, sim=RandomSimilarityProvider(queryNorm=true,coord=no): {}, locale=fi_FI, timezone=America/Caracas
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_25 (64-bit)/cpus=16,threads=70,free=145633128,total=321912832
   [junit4]   2> NOTE: All tests run in this JVM: [TermsComponentTest, JSONWriterTest, DeleteReplicaTest, SOLR749Test, SolrXmlInZkTest, TestReloadAndDeleteDocs, TestFieldTypeResource, RAMDirectoryFactoryTest, ParsingFieldUpdateProcessorsTest, SimpleFacetsTest, TestComponentsName, TestOmitPositions, TestIndexSearcher, TimeZoneUtilsTest, SolrCoreTest, ShowFileRequestHandlerTest, LeaderElectionIntegrationTest, TestCopyFieldCollectionResource, CursorPagingTest, TestNonDefinedSimilarityFactory, HdfsBasicDistributedZk2Test, WordBreakSolrSpellCheckerTest, ChaosMonkeySafeLeaderTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeySafeLeaderTest -Dtests.seed=4D75887B35EB759B -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fi_FI -Dtests.timezone=America/Caracas -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | ChaosMonkeySafeLeaderTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4D75887B35EB759B]:0)
   [junit4] Completed on J1 in 7224.58s, 1 test, 2 errors <<< FAILURES!

[...truncated 213 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:471: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:451: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:490: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1278: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:911: There were test failures: 363 suites, 1565 tests, 1 suite-level error, 2 errors, 228 ignored (7 assumptions)

Total time: 151 minutes 55 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1910 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1910/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.BasicDistributedZkTest.testDistribSearch

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:25648/collection1

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:25648/collection1
	at __randomizedtesting.SeedInfo.seed([57BED07FA9EB55CB:D6585E67DEB435F7]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:505)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:199)
	at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:117)
	at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:168)
	at org.apache.solr.client.solrj.SolrServer.commit(SolrServer.java:146)
	at org.apache.solr.cloud.BasicDistributedZkTest.testUpdateProcessorsRunOnlyOnce(BasicDistributedZkTest.java:663)
	at org.apache.solr.cloud.BasicDistributedZkTest.doTest(BasicDistributedZkTest.java:372)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:870)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
	at java.lang.Thread.run(Thread.java:724)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:150)
	at java.net.SocketInputStream.read(SocketInputStream.java:121)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
	at sun.security.ssl.InputRecord.read(InputRecord.java:480)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:682)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:486)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:395)
	... 47 more




Build Log:
[...truncated 10145 lines...]
   [junit4] JVM J1: stdout was not empty, see: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/temp/junit4-J1-20140225_055110_474.sysout
   [junit4] >>> JVM J1: stdout (verbatim) ----
   [junit4] #
   [junit4] # A fatal error has been detected by the Java Runtime Environment:
   [junit4] #
   [junit4] #  Internal Error (assembler_x86.inline.hpp:46), pid=3414, tid=34378089472
   [junit4] #  guarantee(this->is8bit(imm8)) failed: Short forward jump exceeds 8-bit offset
   [junit4] #
   [junit4] # JRE version: 7.0-b15
   [junit4] # Java VM: OpenJDK 64-Bit Server VM (23.21-b01 mixed mode bsd-amd64 compressed oops)
   [junit4] # Core dump written. Default location: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/core or core.3414
   [junit4] #
   [junit4] # An error report file with more information is saved as:
   [junit4] # /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/hs_err_pid3414.log
   [junit4] #
   [junit4] # If you would like to submit a bug report, please visit:
   [junit4] #   http://bugreport.sun.com/bugreport/crash.jsp
   [junit4] #
   [junit4] <<< JVM J1: EOF ----

[...truncated 44 lines...]
   [junit4] Suite: org.apache.solr.cloud.BasicDistributedZkTest
   [junit4]   2> 1860106 T2434 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 1860106 T2434 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 1860113 T2434 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solrtest-BasicDistributedZkTest-1393309333886
   [junit4]   2> 1860115 T2434 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   2> 1860115 T2435 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1860216 T2434 oasc.ZkTestServer.run start zk server on port:56655
   [junit4]   2> 1860217 T2434 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1860222 T2441 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@c4d0ac5 name:ZooKeeperConnection Watcher:127.0.0.1:56655 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1860222 T2434 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1860223 T2434 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1860248 T2434 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1860251 T2443 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@10e50619 name:ZooKeeperConnection Watcher:127.0.0.1:56655/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1860251 T2434 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1860251 T2434 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 1860257 T2434 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 1860260 T2434 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 1860269 T2434 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 1860273 T2434 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1860274 T2434 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1860288 T2434 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1860289 T2434 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 1860395 T2434 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1860395 T2434 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1860399 T2434 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1860400 T2434 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1860403 T2434 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1860404 T2434 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1860408 T2434 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1860409 T2434 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 1860412 T2434 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1860413 T2434 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1860416 T2434 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1860417 T2434 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1860420 T2434 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1860421 T2434 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1860424 T2434 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1860425 T2434 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1860428 T2434 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1860429 T2434 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1860434 T2434 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1860442 T2445 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@467f0c23 name:ZooKeeperConnection Watcher:127.0.0.1:56655/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1860443 T2434 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1861424 T2434 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1861428 T2434 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 1861430 T2434 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:41824
   [junit4]   2> 1861432 T2434 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1861433 T2434 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1861433 T2434 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393309334218
   [junit4]   2> 1861434 T2434 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393309334218/'
   [junit4]   2> 1861459 T2434 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393309334218/solr.xml
   [junit4]   2> 1861525 T2434 oasc.CoreContainer.<init> New CoreContainer 721619428
   [junit4]   2> 1861526 T2434 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393309334218/]
   [junit4]   2> 1861528 T2434 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1861528 T2434 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1861528 T2434 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1861528 T2434 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1861529 T2434 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1861529 T2434 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1861529 T2434 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1861530 T2434 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1861530 T2434 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1861541 T2434 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1861541 T2434 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1861541 T2434 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1861542 T2434 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56655/solr
   [junit4]   2> 1861542 T2434 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1861543 T2434 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1861546 T2456 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f2b53dd name:ZooKeeperConnection Watcher:127.0.0.1:56655 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1861547 T2434 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1861550 T2434 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1861565 T2458 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a294371 name:ZooKeeperConnection Watcher:127.0.0.1:56655/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1861565 T2434 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1861568 T2434 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1861573 T2434 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 1861582 T2434 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 1861585 T2434 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41824_
   [junit4]   2> 1861587 T2434 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:41824_
   [junit4]   2> 1861591 T2434 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 1861594 T2434 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 1861608 T2434 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:41824_
   [junit4]   2> 1861608 T2434 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1861611 T2434 oasc.Overseer.start Overseer (id=91311920506011652-127.0.0.1:41824_-n_0000000000) starting
   [junit4]   2> 1861620 T2434 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 1861632 T2460 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1861633 T2434 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 1861633 T2460 oasc.OverseerCollectionProcessor.prioritizeOverseerNodes prioritizing overseer nodes
   [junit4]   2> 1861635 T2434 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 1861643 T2434 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1861645 T2459 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1861652 T2461 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 1861653 T2461 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1861654 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1861654 T2461 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1861655 T2459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1861656 T2459 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:41824",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:41824_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1861657 T2459 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 1861657 T2459 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1861660 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1861668 T2458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1862656 T2461 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1862656 T2461 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393309334218/collection1
   [junit4]   2> 1862656 T2461 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 1862657 T2461 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1862658 T2461 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 1862659 T2461 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1862659 T2461 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393309334218/collection1/'
   [junit4]   2> 1862661 T2461 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393309334218/collection1/lib/.svn/' to classloader
   [junit4]   2> 1862661 T2461 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393309334218/collection1/lib/classes/' to classloader
   [junit4]   2> 1862662 T2461 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393309334218/collection1/lib/README' to classloader
   [junit4]   2> 1862736 T2461 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 1862809 T2461 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1862911 T2461 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1862923 T2461 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1863582 T2461 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1863583 T2461 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1863584 T2461 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1863597 T2461 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1863602 T2461 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1863619 T2461 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1863625 T2461 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1863629 T2461 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1863631 T2461 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1863631 T2461 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1863631 T2461 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1863633 T2461 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1863633 T2461 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1863633 T2461 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1863634 T2461 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.BasicDistributedZkTest-controljetty-1393309334218/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/control/data/
   [junit4]   2> 1863634 T2461 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dbbeee7
   [junit4]   2> 1863635 T2461 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/control/data
   [junit4]   2> 1863635 T2461 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/control/data/index/
   [junit4]   2> 1863635 T2461 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1863636 T2461 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/control/data/index
   [junit4]   2> 1863636 T2461 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1045610588, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1863637 T2461 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6768fcfb lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7787363d),segFN=segments_1,generation=1}
   [junit4]   2> 1863638 T2461 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1863642 T2461 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1863643 T2461 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1863643 T2461 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1863643 T2461 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1863644 T2461 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1863644 T2461 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1863645 T2461 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1863645 T2461 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1863645 T2461 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1863646 T2461 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1863647 T2461 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1863647 T2461 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1863647 T2461 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1863648 T2461 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1863648 T2461 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1863649 T2461 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1863665 T2461 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1863672 T2461 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1863672 T2461 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1863673 T2461 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.793231173948885]
   [junit4]   2> 1863673 T2461 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6768fcfb lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7787363d),segFN=segments_1,generation=1}
   [junit4]   2> 1863674 T2461 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1863674 T2461 oass.SolrIndexSearcher.<init> Opening Searcher@29b7a602[collection1] main
   [junit4]   2> 1863678 T2462 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@29b7a602[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1863680 T2461 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1863681 T2434 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 1863681 T2434 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1863681 T2465 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:41824 collection:control_collection shard:shard1
   [junit4]   2> 1863684 T2465 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1863687 T2434 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1863689 T2467 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3acbaef9 name:ZooKeeperConnection Watcher:127.0.0.1:56655/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1863689 T2434 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1863704 T2434 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1863708 T2465 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 1863708 T2434 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1863711 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863711 T2465 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1863711 T2465 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C412 name=collection1 org.apache.solr.core.SolrCore@2610515e url=https://127.0.0.1:41824/collection1 node=127.0.0.1:41824_ C412_STATE=coll:control_collection core:collection1 props:{state=down, base_url=https://127.0.0.1:41824, core=collection1, node_name=127.0.0.1:41824_}
   [junit4]   2> 1863712 T2465 C412 P41824 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:41824/collection1/
   [junit4]   2> 1863712 T2465 C412 P41824 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1863712 T2465 C412 P41824 oasc.SyncStrategy.syncToMe https://127.0.0.1:41824/collection1/ has no replicas
   [junit4]   2> 1863712 T2459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1863713 T2465 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:41824/collection1/ shard1
   [junit4]   2> 1863713 T2465 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1863722 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863723 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1863723 T2458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1863742 T2459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1863746 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863849 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1863849 T2458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1863895 T2465 oasc.ZkController.register We are https://127.0.0.1:41824/collection1/ and leader is https://127.0.0.1:41824/collection1/
   [junit4]   2> 1863895 T2465 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:41824
   [junit4]   2> 1863895 T2465 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1863895 T2465 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 1863895 T2465 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1863897 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863897 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863898 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1863898 T2465 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1863899 T2459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1863900 T2459 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:41824",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:41824_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 1863909 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1864014 T2458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1864014 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1864722 T2434 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 1864723 T2434 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1864727 T2434 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 1864728 T2434 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:25648
   [junit4]   2> 1864730 T2434 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1864731 T2434 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1864731 T2434 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393309337481
   [junit4]   2> 1864731 T2434 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393309337481/'
   [junit4]   2> 1864757 T2434 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393309337481/solr.xml
   [junit4]   2> 1864851 T2434 oasc.CoreContainer.<init> New CoreContainer 272096330
   [junit4]   2> 1864852 T2434 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393309337481/]
   [junit4]   2> 1864854 T2434 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1864854 T2434 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1864855 T2434 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1864855 T2434 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1864856 T2434 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1864856 T2434 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1864857 T2434 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1864857 T2434 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1864858 T2434 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1864873 T2434 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1864874 T2434 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1864874 T2434 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1864875 T2434 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56655/solr
   [junit4]   2> 1864875 T2434 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1864876 T2434 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1864879 T2478 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a4bd3e name:ZooKeeperConnection Watcher:127.0.0.1:56655 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1864880 T2434 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1864899 T2434 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1864901 T2480 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b9c6122 name:ZooKeeperConnection Watcher:127.0.0.1:56655/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1864901 T2434 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1864908 T2434 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1865911 T2434 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:25648_
   [junit4]   2> 1865914 T2434 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:25648_
   [junit4]   2> 1865918 T2458 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1865918 T2480 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1865918 T2467 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1865929 T2481 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 1865929 T2481 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1865931 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1865931 T2481 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1865931 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1865931 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1865932 T2459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1865933 T2459 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:25648",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:25648_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1865934 T2459 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 1865934 T2459 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 1865938 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1865939 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1865939 T2480 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1865939 T2458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1866932 T2481 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1866932 T2481 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393309337481/collection1
   [junit4]   2> 1866932 T2481 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 1866934 T2481 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1866934 T2481 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1866935 T2481 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1866935 T2481 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393309337481/collection1/'
   [junit4]   2> 1866937 T2481 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393309337481/collection1/lib/.svn/' to classloader
   [junit4]   2> 1866937 T2481 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393309337481/collection1/lib/README' to classloader
   [junit4]   2> 1866938 T2481 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393309337481/collection1/lib/classes/' to classloader
   [junit4]   2> 1867011 T2481 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 1867085 T2481 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1867188 T2481 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1867200 T2481 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1867858 T2481 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1867859 T2481 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1867860 T2481 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1867874 T2481 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1867878 T2481 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1867896 T2481 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1867901 T2481 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1867907 T2481 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1867908 T2481 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1867909 T2481 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1867909 T2481 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1867910 T2481 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1867911 T2481 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1867911 T2481 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1867911 T2481 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.BasicDistributedZkTest-jetty1-1393309337481/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/jetty1/
   [junit4]   2> 1867912 T2481 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dbbeee7
   [junit4]   2> 1867913 T2481 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/jetty1
   [junit4]   2> 1867913 T2481 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/jetty1/index/
   [junit4]   2> 1867913 T2481 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 1867914 T2481 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/jetty1/index
   [junit4]   2> 1867914 T2481 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1045610588, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1867915 T2481 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@4e48b3db lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2c1a71ac),segFN=segments_1,generation=1}
   [junit4]   2> 1867916 T2481 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1867921 T2481 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1867922 T2481 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1867922 T2481 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1867923 T2481 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1867923 T2481 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1867923 T2481 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1867924 T2481 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1867924 T2481 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1867924 T2481 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1867925 T2481 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1867926 T2481 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1867926 T2481 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1867927 T2481 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1867927 T2481 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1867928 T2481 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1867929 T2481 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1867945 T2481 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1867951 T2481 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1867951 T2481 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1867952 T2481 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.793231173948885]
   [junit4]   2> 1867953 T2481 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@4e48b3db lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2c1a71ac),segFN=segments_1,generation=1}
   [junit4]   2> 1867953 T2481 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1867954 T2481 oass.SolrIndexSearcher.<init> Opening Searcher@a4522e7[collection1] main
   [junit4]   2> 1867959 T2482 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@a4522e7[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1867962 T2481 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1867963 T2434 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 1867964 T2434 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1867963 T2485 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:25648 collection:collection1 shard:shard2
   [junit4]   2> 1867966 T2485 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 1868010 T2485 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 1868019 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1868019 T2485 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1868020 T2485 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C413 name=collection1 org.apache.solr.core.SolrCore@6c575b29 url=https://127.0.0.1:25648/collection1 node=127.0.0.1:25648_ C413_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:25648, core=collection1, node_name=127.0.0.1:25648_}
   [junit4]   2> 1868020 T2485 C413 P25648 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:25648/collection1/
   [junit4]   2> 1868020 T2485 C413 P25648 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1868021 T2485 C413 P25648 oasc.SyncStrategy.syncToMe https://127.0.0.1:25648/collection1/ has no replicas
   [junit4]   2> 1868021 T2459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1868021 T2485 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:25648/collection1/ shard2
   [junit4]   2> 1868021 T2485 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 1868034 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1868036 T2458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1868036 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1868036 T2480 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1868041 T2459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1868046 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1868150 T2458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1868150 T2480 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1868150 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1868193 T2485 oasc.ZkController.register We are https://127.0.0.1:25648/collection1/ and leader is https://127.0.0.1:25648/collection1/
   [junit4]   2> 1868193 T2485 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:25648
   [junit4]   2> 1868193 T2485 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1868193 T2485 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 1868193 T2485 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1868195 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1868195 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1868195 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1868196 T2485 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1868197 T2459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1868198 T2459 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:25648",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:25648_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 1868207 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1868310 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1868310 T2480 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1868310 T2458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1868915 T2434 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 1868916 T2434 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1868920 T2434 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 1868921 T2434 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:46948
   [junit4]   2> 1868923 T2434 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1868924 T2434 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1868924 T2434 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393309341739
   [junit4]   2> 1868924 T2434 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393309341739/'
   [junit4]   2> 1868950 T2434 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393309341739/solr.xml
   [junit4]   2> 1869015 T2434 oasc.CoreContainer.<init> New CoreContainer 244843597
   [junit4]   2> 1869016 T2434 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393309341739/]
   [junit4]   2> 1869017 T2434 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1869017 T2434 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1869018 T2434 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1869018 T2434 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1869018 T2434 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1869018 T2434 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1869019 T2434 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1869019 T2434 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1869019 T2434 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1869030 T2434 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1869030 T2434 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1869030 T2434 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1869031 T2434 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56655/solr
   [junit4]   2> 1869031 T2434 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1869032 T2434 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1869061 T2496 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@62c8a780 name:ZooKeeperConnection Watcher:127.0.0.1:56655 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1869061 T2434 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1869071 T2434 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1869073 T2498 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5579b45f name:ZooKeeperConnection Watcher:127.0.0.1:56655/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1869073 T2434 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1869080 T2434 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1870083 T2434 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46948_
   [junit4]   2> 1870094 T2434 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46948_
   [junit4]   2> 1870098 T2467 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1870098 T2458 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1870099 T2498 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1870098 T2480 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1870116 T2499 oasc.ZkController.publish publishing core=collection1 state=down
   [junit4]   2> 1870116 T2499 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1870118 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1870118 T2499 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1870118 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1870118 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1870120 T2459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1870121 T2459 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:46948",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:46948_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1870121 T2459 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 1870121 T2459 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1870125 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1870127 T2498 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1870127 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1870127 T2458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1870127 T2480 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1871119 T2499 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1871119 T2499 oasc.CoreContainer.create Creating SolrCore 'collection1' using instanceDir: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393309341739/collection1
   [junit4]   2> 1871119 T2499 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 1871120 T2499 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1871120 T2499 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1871122 T2499 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1871122 T2499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393309341739/collection1/'
   [junit4]   2> 1871123 T2499 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393309341739/collection1/lib/.svn/' to classloader
   [junit4]   2> 1871124 T2499 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393309341739/collection1/lib/classes/' to classloader
   [junit4]   2> 1871125 T2499 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393309341739/collection1/lib/README' to classloader
   [junit4]   2> 1871197 T2499 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_48
   [junit4]   2> 1871270 T2499 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1871372 T2499 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1871383 T2499 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1872049 T2499 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1872051 T2499 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1872051 T2499 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1872065 T2499 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1872070 T2499 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1872090 T2499 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1872095 T2499 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1872100 T2499 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1872102 T2499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1872102 T2499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1872102 T2499 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1872103 T2499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1872104 T2499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1872104 T2499 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1872104 T2499 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./org.apache.solr.cloud.BasicDistributedZkTest-jetty2-1393309341739/collection1/, dataDir=./org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/jetty2/
   [junit4]   2> 1872105 T2499 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@6dbbeee7
   [junit4]   2> 1872106 T2499 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/jetty2
   [junit4]   2> 1872106 T2499 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/jetty2/index/
   [junit4]   2> 1872107 T2499 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 1872107 T2499 oasc.CachingDirectoryFactory.get return new directory for ./org.apache.solr.cloud.BasicDistributedZkTest-1393309333886/jetty2/index
   [junit4]   2> 1872108 T2499 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1045610588, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1872109 T2499 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@9a93c33 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14a81561),segFN=segments_1,generation=1}
   [junit4]   2> 1872109 T2499 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1872115 T2499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1872115 T2499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1872115 T2499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1872116 T2499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1872116 T2499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1872116 T2499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1872117 T2499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1872117 T2499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1872118 T2499 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1872132 T2499 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1872133 T2499 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1872133 T2499 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1872134 T2499 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1872135 T2499 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1872135 T2499 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1872136 T2499 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1872153 T2499 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1872158 T2499 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1872159 T2499 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1872159 T2499 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=24, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.793231173948885]
   [junit4]   2> 1872160 T2499 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@9a93c33 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@14a81561),segFN=segments_1,generation=1}
   [junit4]   2> 1872161 T2499 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1872161 T2499 oass.SolrIndexSearcher.<init> Opening Searcher@3cd58ba7[collection1] main
   [junit4]   2> 1872166 T2500 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@3cd58ba7[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1872169 T2499 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1872170 T2434 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 1872171 T2434 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1872170 T2503 oasc.ZkController.register Register replica - core:collection1 address:https://127.0.0.1:46948 collection:collection1 shard:shard1
   [junit4]   2> 1872173 T2503 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1872202 T2503 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 1872213 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1872214 T2503 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1872214 T2503 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C414 name=collection1 org.apache.solr.core.SolrCore@3094b691 url=https://127.0.0.1:46948/collection1 node=127.0.0.1:46948_ C414_STATE=coll:collection1 core:collection1 props:{state=down, base_url=https://127.0.0.1:46948, core=collection1, node_name=127.0.0.1:46948_}
   [junit4]   2> 1872214 T2503 C414 P46948 oasc.SyncStrategy.sync Sync replicas to https://127.0.0.1:46948/collection1/
   [junit4]   2> 1872214 T2503 C414 P46948 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1872215 T2503 C414 P46948 oasc.SyncStrategy.syncToMe https://127.0.0.1:46948/collection1/ has no replicas
   [junit4]   2> 1872215 T2459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1872215 T2503 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: https://127.0.0.1:46948/collection1/ shard1
   [junit4]   2> 1872215 T2503 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1872219 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1872229 T2458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1872229 T2498 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1872229 T2480 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1872229 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1872240 T2459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1872244 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1872351 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1872351 T2498 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1872351 T2480 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1872351 T2458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1872392 T2503 oasc.ZkController.register We are https://127.0.0.1:46948/collection1/ and leader is https://127.0.0.1:46948/collection1/
   [junit4]   2> 1872392 T2503 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:46948
   [junit4]   2> 1872392 T2503 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1872392 T2503 oasc.ZkController.publish publishing core=collection1 state=active
   [junit4]   2> 1872392 T2503 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1872394 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1872394 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1872394 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1872395 T2503 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1872396 T2459 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1872397 T2459 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:46948",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:46948_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 1872400 T2458 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1872507 T2480 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1872507 T2458 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1872507 T2467 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1872507 T2498 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1873046 T2434 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 1873048 T2434 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1873052 T2434 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 1873053 T2434 oejs.AbstractConnector.doStart Started SslSocketConnector@127.0.0.1:61460
   [junit4]   2> 1873056 T2434 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1873056 T2434 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1873056 T2434 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393309345946
   [junit4]   2> 1873057 T2434 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393309345946/'
   [junit4]   2> 1873082 T2434 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393309345946/solr.xml
   [junit4]   2> 1873148 T2434 oasc.CoreContainer.<init> New CoreContainer 1850918236
   [junit4]   2> 1873149 T2434 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./org.apache.solr.cloud.BasicDistributedZkTest-jetty3-1393309345946/]
   [junit4]   2> 1873150 T2434 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1873150 T2434 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1873150 T2434 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1873151 T2434 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1873151 T2434 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1873151 T2434 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1873152 T2434 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1873152 T2434 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1873152 T2434 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1873161 T2434 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1873162 T2434 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1873162 T2434 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1873163 T2434 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:56655/solr
   [junit4]   2> 1873163 T2434 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1873164 T2434 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1873194 T2514 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@227fcf6b name:ZooKeeperConnection Watcher:127.0.0.1:56655 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1873194 T2434 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1873198 T2434 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1873207 T2516 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8cc558e name:ZooKeeperConnection Watcher:127.0.0.1:56655/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1873208 T2434 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1873214 T2434 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1874218 T2434 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61460_
   [junit4]   2> 1874227 T2434 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.

[...truncated too long message...]

r:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/lib/spatial4j-0.4.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/solrj/lib/commons-io-2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/solrj/lib/httpclient-4.3.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/solrj/lib/httpcore-4.3.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/solrj/lib/httpmime-4.3.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/solrj/lib/jcl-over-slf4j-1.7.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/solrj/lib/jul-to-slf4j-1.7.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/solrj/lib/log4j-1.2.16.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/solrj/lib/noggit-0.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/solrj/lib/slf4j-api-1.7.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/solrj/lib/slf4j-log4j12-1.7.6.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/solrj/lib/wstx-asl-3.2.7.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/solrj/lib/zookeeper-3.4.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/jetty-continuation-8.1.10.v20130312.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/jetty-deploy-8.1.10.v20130312.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/jetty-http-8.1.10.v20130312.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/jetty-io-8.1.10.v20130312.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/jetty-jmx-8.1.10.v20130312.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/jetty-security-8.1.10.v20130312.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/jetty-server-8.1.10.v20130312.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/jetty-servlet-8.1.10.v20130312.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/jetty-util-8.1.10.v20130312.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/jetty-webapp-8.1.10.v20130312.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/jetty-xml-8.1.10.v20130312.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/servlet-api-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/example-DIH/solr/db/lib/hsqldb-1.8.0.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/build/core/classes/java:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/test-framework/lib/junit-4.10.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/test-framework/lib/randomizedtesting-runner-2.0.13.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/antlr-runtime-3.5.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/asm-4.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/asm-commons-4.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/cglib-nodep-2.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/commons-collections-3.2.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/dom4j-1.6.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/easymock-3.0.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/hadoop-common-2.2.0-tests.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/hadoop-hdfs-2.2.0-tests.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/hppc-0.5.2.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/javax.servlet-api-3.0.1.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/jersey-core-1.8.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/jetty-6.1.26.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/jetty-util-6.1.26.jar:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/test-lib/objenesis-1.2.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/home/hudson/.ant/lib/ivy-2.3.0.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/hudson/hudson-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/usr/local/openjdk7/lib/tools.jar:/home/hudson/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.0.13.jar -ea:org.apache.lucene... -ea:org.apache.solr... com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/temp/junit4-J1-20140225_055110_474.events @/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/temp/junit4-J1-20140225_055110_474.suites
   [junit4] ERROR: JVM J1 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1373)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:119)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:911)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:908)
   [junit4] 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4] 	at java.lang.Thread.run(Thread.java:724)

BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:471: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:451: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:490: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1278: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:911: At least one slave process threw an exception, first: Forked process returned with error code: 134. Very likely a JVM crash.  Process output piped in logs above.

Total time: 105 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure