You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2014/07/02 01:44:36 UTC

[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2017 - Still Failing

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2017/

All tests passed

Build Log:
[...truncated 29270 lines...]
check-licenses:
     [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/log4j-1.2.16.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/log4j-1.2.16.jar.sha1

[...truncated 1 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:70: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.

Total time: 137 minutes 25 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-4.x-Java7 #2008
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 20 ms
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



Re: [JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2024 - Still Failing

Posted by Steve Rowe <sa...@gmail.com>.
I logged into lucene.zones.apache.org and ran ‘ant clean-jars’ as the hudson user in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/ - hopefully that’ll allow this job to succeed one of these days...

Steve

On Jul 6, 2014, at 6:26 PM, Apache Jenkins Server <je...@builds.apache.org> wrote:

> Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2024/
> 
> All tests passed
> 
> Build Log:
> [...truncated 29221 lines...]
> check-licenses:
>     [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
> [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/log4j-1.2.16.jar
> [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/log4j-1.2.16.jar.sha1
> 
> [...truncated 1 lines...]
> BUILD FAILED
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:467: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:70: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.
> 
> Total time: 133 minutes 23 seconds
> Build step 'Invoke Ant' marked build as failure
> Archiving artifacts
> Sending artifact delta relative to Lucene-Solr-Tests-4.x-Java7 #2008
> Archived 1 artifacts
> Archive block size is 32768
> Received 0 blocks and 464 bytes
> Compression is 0.0%
> Took 82 ms
> Recording test results
> Email was triggered for: Failure
> Sending email for trigger: Failure
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2024 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2024/

All tests passed

Build Log:
[...truncated 29221 lines...]
check-licenses:
     [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/log4j-1.2.16.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/log4j-1.2.16.jar.sha1

[...truncated 1 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:70: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.

Total time: 133 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-4.x-Java7 #2008
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 82 ms
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2023 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2023/

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup:     1) Thread[id=11173, name=Thread-4015, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]         at java.net.PlainSocketImpl.socketConnect(Native Method)         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)         at java.net.Socket.connect(Socket.java:579)         at java.net.Socket.connect(Socket.java:528)         at sun.net.NetworkClient.doConnect(NetworkClient.java:180)         at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)         at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)         at java.net.URL.openStream(URL.java:1037)         at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   1) Thread[id=11173, name=Thread-4015, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at java.net.Socket.connect(Socket.java:528)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
        at java.net.URL.openStream(URL.java:1037)
        at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
	at __randomizedtesting.SeedInfo.seed([DA4A152CC2DEC7CC]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=11173, name=Thread-4015, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]         at java.net.PlainSocketImpl.socketConnect(Native Method)         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)         at java.net.Socket.connect(Socket.java:579)         at java.net.Socket.connect(Socket.java:528)         at sun.net.NetworkClient.doConnect(NetworkClient.java:180)         at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)         at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)         at java.net.URL.openStream(URL.java:1037)         at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=11173, name=Thread-4015, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at java.net.Socket.connect(Socket.java:528)
        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
        at java.net.URL.openStream(URL.java:1037)
        at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
	at __randomizedtesting.SeedInfo.seed([DA4A152CC2DEC7CC]:0)




Build Log:
[...truncated 11687 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandlerBackup
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/init-core-data-001
   [junit4]   2> 3956610 T11153 oas.SolrTestCaseJ4.setUp ###Starting doTestBackup
   [junit4]   2> 3956623 T11153 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 3956624 T11153 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:44958
   [junit4]   2> 3956625 T11153 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 3956625 T11153 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 3956625 T11153 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001
   [junit4]   2> 3956625 T11153 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/'
   [junit4]   2> 3956662 T11153 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/solr.xml
   [junit4]   2> 3956719 T11153 oasc.CoreContainer.<init> New CoreContainer 386757685
   [junit4]   2> 3956719 T11153 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/]
   [junit4]   2> 3956720 T11153 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 3956721 T11153 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 3956721 T11153 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 3956721 T11153 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 3956721 T11153 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 3956722 T11153 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 3956722 T11153 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 3956722 T11153 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 3956722 T11153 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 3956723 T11153 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3956723 T11153 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3956723 T11153 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 3956728 T11163 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/'
   [junit4]   2> 3956763 T11163 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
   [junit4]   2> 3956794 T11163 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3956795 T11163 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 3956800 T11163 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 3956812 T11163 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 3956813 T11163 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir ./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/
   [junit4]   2> 3956813 T11163 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 3956814 T11163 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/data/
   [junit4]   2> 3956814 T11163 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 3956814 T11163 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/data
   [junit4]   2> 3956814 T11163 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/data/index/
   [junit4]   2> 3956815 T11163 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 3956815 T11163 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/data/index
   [junit4]   2> 3956815 T11163 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=42, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=52.37109375, floorSegmentMB=0.865234375, forceMergeDeletesPctAllowed=12.944192703201649, segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 3956816 T11163 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@538ab657 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3a67eda1),segFN=segments_1,generation=1}
   [junit4]   2> 3956816 T11163 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3956817 T11163 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 3956818 T11163 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 3956818 T11163 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 3956818 T11163 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 3956818 T11163 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 3956818 T11163 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 3956819 T11163 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 3956832 T11163 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 3956833 T11163 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 3956834 T11163 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 3956834 T11163 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=22.6630859375, floorSegmentMB=1.927734375, forceMergeDeletesPctAllowed=12.568101809057882, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 3956834 T11163 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@538ab657 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3a67eda1),segFN=segments_1,generation=1}
   [junit4]   2> 3956835 T11163 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 3956835 T11163 oass.SolrIndexSearcher.<init> Opening Searcher@595a2caf[collection1] main
   [junit4]   2> 3956835 T11163 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/conf
   [junit4]   2> 3956835 T11163 oasr.RestManager.init Initializing RestManager with initArgs: {storageDir=./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/conf/}
   [junit4]   2> 3956835 T11163 oasr.ManagedResourceStorage.load Reading _rest_managed.json using file:dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/conf
   [junit4]   2> 3956836 T11163 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 3956836 T11163 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 3956836 T11163 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 3956837 T11163 oash.ReplicationHandler.inform Replication enabled for following config files: schema-replication2.xml:schema.xml
   [junit4]   2> 3956837 T11163 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 3956837 T11163 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 3956837 T11164 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@595a2caf[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 3956838 T11153 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 3956838 T11153 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C1687 name=collection1 org.apache.solr.core.SolrCore@1060bb08
   [junit4]   2> 3956840 T11156 C1687 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 3956841 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {deleteByQuery=*:*} 0 1
   [junit4]   2> 3956844 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[0]} 0 0
   [junit4]   2> 3956847 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[1]} 0 0
   [junit4]   2> 3956850 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[2]} 0 0
   [junit4]   2> 3956852 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[3]} 0 0
   [junit4]   2> 3956854 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[4]} 0 0
   [junit4]   2> 3956856 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[5]} 0 0
   [junit4]   2> 3956858 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[6]} 0 0
   [junit4]   2> 3956860 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[7]} 0 0
   [junit4]   2> 3956862 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[8]} 0 0
   [junit4]   2> 3956864 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[9]} 0 0
   [junit4]   2> 3956866 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[10]} 0 0
   [junit4]   2> 3956868 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[11]} 0 0
   [junit4]   2> 3956870 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[12]} 0 0
   [junit4]   2> 3956872 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[13]} 0 0
   [junit4]   2> 3956874 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[14]} 0 0
   [junit4]   2> 3956876 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[15]} 0 0
   [junit4]   2> 3956878 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[16]} 0 0
   [junit4]   2> 3956880 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[17]} 0 0
   [junit4]   2> 3956882 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[18]} 0 0
   [junit4]   2> 3956884 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[19]} 0 0
   [junit4]   2> 3956886 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[20]} 0 0
   [junit4]   2> 3956888 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[21]} 0 0
   [junit4]   2> 3956890 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[22]} 0 0
   [junit4]   2> 3956892 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[23]} 0 0
   [junit4]   2> 3956894 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[24]} 0 0
   [junit4]   2> 3956896 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[25]} 0 0
   [junit4]   2> 3956898 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[26]} 0 0
   [junit4]   2> 3956900 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[27]} 0 0
   [junit4]   2> 3956902 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[28]} 0 0
   [junit4]   2> 3956904 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[29]} 0 0
   [junit4]   2> 3956906 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[30]} 0 0
   [junit4]   2> 3956908 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[31]} 0 0
   [junit4]   2> 3956910 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[32]} 0 0
   [junit4]   2> 3956912 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[33]} 0 0
   [junit4]   2> 3956915 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[34]} 0 0
   [junit4]   2> 3956917 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[35]} 0 0
   [junit4]   2> 3956919 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[36]} 0 0
   [junit4]   2> 3956921 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[37]} 0 0
   [junit4]   2> 3956923 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[38]} 0 0
   [junit4]   2> 3956925 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[39]} 0 0
   [junit4]   2> 3956928 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[40]} 0 0
   [junit4]   2> 3956930 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[41]} 0 0
   [junit4]   2> 3956932 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[42]} 0 0
   [junit4]   2> 3956934 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[43]} 0 0
   [junit4]   2> 3956936 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[44]} 0 0
   [junit4]   2> 3956938 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[45]} 0 0
   [junit4]   2> 3956940 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[46]} 0 0
   [junit4]   2> 3956942 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[47]} 0 0
   [junit4]   2> 3956944 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[48]} 0 0
   [junit4]   2> 3956946 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[49]} 0 0
   [junit4]   2> 3956948 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[50]} 0 0
   [junit4]   2> 3956950 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[51]} 0 0
   [junit4]   2> 3956952 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[52]} 0 0
   [junit4]   2> 3956954 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[53]} 0 0
   [junit4]   2> 3956956 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[54]} 0 0
   [junit4]   2> 3956958 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[55]} 0 0
   [junit4]   2> 3956960 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin} {add=[56]} 0 0
   [junit4]   2> 3956962 T11156 C1687 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 3956964 T11156 C1687 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@538ab657 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3a67eda1),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@538ab657 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3a67eda1),segFN=segments_2,generation=2}
   [junit4]   2> 3956964 T11156 C1687 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 3957017 T11156 C1687 oass.SolrIndexSearcher.<init> Opening Searcher@289045d4[collection1] main
   [junit4]   2> 3957017 T11156 C1687 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 3957017 T11164 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@289045d4[collection1] main{StandardDirectoryReader(segments_2:4:nrt _0(4.10):c57)}
   [junit4]   2> 3957018 T11156 C1687 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={version=2&wt=javabin&softCommit=false&commit=true&waitSearcher=true} {commit=} 0 56
   [junit4]   2> 3957023 T11158 C1687 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 3957030 T11157 C1687 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=g&command=backup} status=0 QTime=8 
   [junit4]   2> 3957030 T11168 oash.SnapShooter.createSnapshot Creating backup snapshot...
   [junit4]   2> 3957038 T11168 oash.SnapShooter.createSnapshot Done creating backup snapshot, completed at: Sat Jul 05 20:23:38 UYT 2014
   [junit4]   2> 3957038 T11168 oash.SnapShooter.createSnapshot Done creating backup snapshot: g
   [junit4]   2> 3957227 T11157 C1687 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 3958239 T11157 C1687 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 3958240 T11158 C1687 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=wgtvoawon&command=backup} status=0 QTime=2 
   [junit4]   2> 3958240 T11170 oash.SnapShooter.createSnapshot Creating backup snapshot...
   [junit4]   2> 3958249 T11170 oash.SnapShooter.createSnapshot Done creating backup snapshot, completed at: Sat Jul 05 20:23:39 UYT 2014
   [junit4]   2> 3958249 T11170 oash.SnapShooter.createSnapshot Done creating backup snapshot: wgtvoawon
   [junit4]   2> 3958441 T11158 C1687 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 3958446 T11158 C1687 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=g&command=deletebackup} status=0 QTime=0 
   [junit4]   2> 3958446 T11172 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: g
   [junit4]   2> 3958448 T11157 C1687 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 3958650 T11157 C1687 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 3958652 T11157 C1687 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 3958652 T11153 oas.SolrTestCaseJ4.tearDown ###Ending doTestBackup
   [junit4]   2> 3958652 T11174 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: wgtvoawon
   [junit4]   2> 3958652 T11158 C1687 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=wgtvoawon&command=deletebackup} status=0 QTime=0 
   [junit4]   2> 3958653 T11153 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=386757685
   [junit4]   2> 3958654 T11153 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/solr,null}
   [junit4]   2> 3958654 T11158 C1687 oasc.SolrException.log ERROR null:org.eclipse.jetty.io.EofException
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142)
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
   [junit4]   2> 		at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
   [junit4]   2> 		at org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:766)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:426)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:136)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2> 		at org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
   [junit4]   2> 		at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 3958655 T11158 C1687 oasc.SolrException.log ERROR null:org.eclipse.jetty.io.EofException
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:142)
   [junit4]   2> 		at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:107)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.implFlushBuffer(StreamEncoder.java:291)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.implFlush(StreamEncoder.java:295)
   [junit4]   2> 		at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:141)
   [junit4]   2> 		at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:229)
   [junit4]   2> 		at org.apache.solr.util.FastWriter.flush(FastWriter.java:137)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.writeResponse(SolrDispatchFilter.java:766)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:426)
   [junit4]   2> 		at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:136)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:229)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.GzipHandler.handle(GzipHandler.java:301)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1077)
   [junit4]   2> 		at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)
   [junit4]   2> 		at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
   [junit4]   2> 		at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
   [junit4]   2> 		at org.eclipse.jetty.server.Server.handle(Server.java:368)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942)
   [junit4]   2> 		at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640)
   [junit4]   2> 		at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
   [junit4]   2> 		at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
   [junit4]   2> 		at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
   [junit4]   2> 		at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
   [junit4]   2> 		at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	
   [junit4]   2> 3958657 T11158 C1687 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1060bb08
   [junit4]   2> 3958657 T11158 C1687 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=57,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 3958657 T11158 C1687 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 3958657 T11158 C1687 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 3958657 T11158 C1687 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 3958658 T11158 C1687 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 3958659 T11158 C1687 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 3958659 T11158 C1687 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/data/index [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/data/index;done=false>>]
   [junit4]   2> 3958659 T11158 C1687 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/data/index
   [junit4]   2> 3958659 T11158 C1687 oasc.CachingDirectoryFactory.closeCacheValue looking to close /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/data [CachedDir<<refCount=0;path=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/data;done=false>>]
   [junit4]   2> 3958660 T11158 C1687 oasc.CachingDirectoryFactory.close Closing directory: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.handler.TestReplicationHandlerBackup-DA4A152CC2DEC7CC-001/solr-instance-001/collection1/data
   [junit4]   2> 3958713 T11153 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> 2110 T11152 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> 22311 T11152 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   [junit4]   2> 	   1) Thread[id=11173, name=Thread-4015, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]   2> 	        at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]   2> 	        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]   2> 	        at java.net.Socket.connect(Socket.java:579)
   [junit4]   2> 	        at java.net.Socket.connect(Socket.java:528)
   [junit4]   2> 	        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]   2> 	        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]   2> 	        at java.net.URL.openStream(URL.java:1037)
   [junit4]   2> 	        at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
   [junit4]   2> 22312 T11152 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=11173, name=Thread-4015, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]   2> 25320 T11152 ccr.ThreadLeakControl.tryToInterruptAll SEVERE There are still zombie threads that couldn't be terminated:
   [junit4]   2> 	   1) Thread[id=11173, name=Thread-4015, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]   2> 	        at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]   2> 	        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]   2> 	        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]   2> 	        at java.net.Socket.connect(Socket.java:579)
   [junit4]   2> 	        at java.net.Socket.connect(Socket.java:528)
   [junit4]   2> 	        at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]   2> 	        at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]   2> 	        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]   2> 	        at java.net.URL.openStream(URL.java:1037)
   [junit4]   2> 	        at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
   [junit4]   2> NOTE: test params are: codec=Lucene45, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=de, timezone=America/Montevideo
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=2,free=245113952,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [DocValuesMultiTest, DisMaxRequestHandlerTest, TestInitQParser, TestRangeQuery, TestCollationField, StressHdfsTest, SoftAutoCommitTest, ChangedSchemaMergeTest, ZkSolrClientTest, HighlighterConfigTest, TestLRUCache, TestOmitPositions, XmlUpdateRequestHandlerTest, DistribCursorPagingTest, TestStressReorder, TestFileDictionaryLookup, TestSimpleQParserPlugin, TestSolrQueryParserResource, DeleteReplicaTest, TestInfoStreamLogging, PathHierarchyTokenizerFactoryTest, TestShardHandlerFactory, PreAnalyzedUpdateProcessorTest, TestNonDefinedSimilarityFactory, FullSolrCloudDistribCmdsTest, SchemaVersionSpecificBehaviorTest, AddBlockUpdateTest, DistribDocExpirationUpdateProcessorTest, DateMathParserTest, TestXIncludeConfig, ClusterStateUpdateTest, SOLR749Test, SuggesterWFSTTest, TestCloudInspectUtil, SolrInfoMBeanTest, AnalysisErrorHandlingTest, DirectUpdateHandlerOptimizeTest, FastVectorHighlighterTest, TestDynamicFieldResource, TestSearcherReuse, TermVectorComponentTest, TestValueSourceCache, ReturnFieldsTest, TestCursorMarkWithoutUniqueKey, TestLuceneMatchVersion, TestFunctionQuery, HdfsWriteToMultipleCollectionsTest, BadComponentTest, RequestHandlersTest, DocExpirationUpdateProcessorFactoryTest, TestTrie, ZkNodePropsTest, DocValuesTest, SystemInfoHandlerTest, TestAtomicUpdateErrorCases, SolrPluginUtilsTest, TestBM25SimilarityFactory, TestStressRecovery, TestReplicationHandler, DistributedQueryComponentOptimizationTest, SpatialFilterTest, ConnectionManagerTest, TestCharFilters, SignatureUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, BadIndexSchemaTest, TestDefaultSimilarityFactory, TestRandomFaceting, TestJoin, TestUtils, RAMDirectoryFactoryTest, TestPseudoReturnFields, SpellPossibilityIteratorTest, TestHashPartitioner, ClusterStateTest, TestSolr4Spatial, SuggesterFSTTest, TimeZoneUtilsTest, TestConfig, TestDistribDocBasedVersion, SolrCmdDistributorTest, TestQuerySenderListener, CoreContainerCoreInitFailuresTest, BlockDirectoryTest, SimpleFacetsTest, TestCollationKeyRangeQueries, TestRandomMergePolicy, TestPartialUpdateDeduplication, TestBlendedInfixSuggestions, ReplicationFactorTest, TestQueryUtils, AnalyticsMergeStrategyTest, TestDynamicFieldCollectionResource, ScriptEngineTest, QueryResultKeyTest, ChaosMonkeyNothingIsSafeTest, RollingRestartTest, BasicDistributedZkTest, MultiThreadedOCPTest, OverseerRolesTest, AliasIntegrationTest, TestCloudManagedSchemaConcurrent, TestCollectionAPI, OpenCloseCoreStressTest, TestShortCircuitedRequests, TestCoreContainer, CursorPagingTest, TestDocBasedVersionConstraints, TestMiniSolrCloudCluster, AutoCommitTest, LeaderElectionTest, TestZkChroot, ShardRoutingCustomTest, CachingDirectoryFactoryTest, TestHighlightDedupGrouping, MergeStrategyTest, DirectUpdateHandlerTest, SolrXmlInZkTest, CoreAdminRequestStatusTest, DistributedDebugComponentTest, ZkControllerTest, ShowFileRequestHandlerTest, TestManagedResourceStorage, HighlighterTest, SolrIndexSplitterTest, DistributedQueryElevationComponentTest, SuggestComponentTest, SpellCheckCollatorTest, TestMultiCoreConfBootstrap, TestExtendedDismaxParser, TestIndexSearcher, TestComplexPhraseQParserPlugin, TestReplicationHandlerBackup]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandlerBackup -Dtests.seed=DA4A152CC2DEC7CC -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=de -Dtests.timezone=America/Montevideo -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J0 | TestReplicationHandlerBackup (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestReplicationHandlerBackup: 
   [junit4]    >    1) Thread[id=11173, name=Thread-4015, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]    >         at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]    >         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]    >         at java.net.Socket.connect(Socket.java:579)
   [junit4]    >         at java.net.Socket.connect(Socket.java:528)
   [junit4]    >         at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]    >         at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]    >         at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]    >         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]    >         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]    >         at java.net.URL.openStream(URL.java:1037)
   [junit4]    >         at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DA4A152CC2DEC7CC]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=11173, name=Thread-4015, state=RUNNABLE, group=TGRP-TestReplicationHandlerBackup]
   [junit4]    >         at java.net.PlainSocketImpl.socketConnect(Native Method)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
   [junit4]    >         at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
   [junit4]    >         at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
   [junit4]    >         at java.net.Socket.connect(Socket.java:579)
   [junit4]    >         at java.net.Socket.connect(Socket.java:528)
   [junit4]    >         at sun.net.NetworkClient.doConnect(NetworkClient.java:180)
   [junit4]    >         at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
   [junit4]    >         at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
   [junit4]    >         at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:652)
   [junit4]    >         at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1323)
   [junit4]    >         at java.net.URL.openStream(URL.java:1037)
   [junit4]    >         at org.apache.solr.handler.TestReplicationHandlerBackup$BackupThread.run(TestReplicationHandlerBackup.java:314)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DA4A152CC2DEC7CC]:0)
   [junit4] Completed on J0 in 25.39s, 1 test, 2 errors <<< FAILURES!

[...truncated 349 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1299: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:923: There were test failures: 405 suites, 1656 tests, 2 suite-level errors, 274 ignored (17 assumptions)

Total time: 111 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-4.x-Java7 #2008
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 186368051 bytes
Compression is 0.0%
Took 32 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2022 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2022/

All tests passed

Build Log:
[...truncated 29260 lines...]
check-licenses:
     [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/log4j-1.2.16.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/log4j-1.2.16.jar.sha1

[...truncated 1 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:70: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.

Total time: 136 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-4.x-Java7 #2008
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 11 ms
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2021 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2021/

3 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIAsyncDistributedZkTest.testDistribSearch

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([84A7796AF29B22FB]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIAsyncDistributedZkTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([84A7796AF29B22FB]:0)


REGRESSION:  org.apache.solr.cloud.HttpPartitionTest.testDistribSearch

Error Message:
No registered leader was found after waiting for 60000ms , collection: c8n_1x3_lf slice: shard1

Stack Trace:
org.apache.solr.common.SolrException: No registered leader was found after waiting for 60000ms , collection: c8n_1x3_lf slice: shard1
	at __randomizedtesting.SeedInfo.seed([84A7796AF29B22FB:541F77285C442C7]:0)
	at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:545)
	at org.apache.solr.cloud.HttpPartitionTest.testRf3WithLeaderFailover(HttpPartitionTest.java:350)
	at org.apache.solr.cloud.HttpPartitionTest.doTest(HttpPartitionTest.java:149)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11188 lines...]
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/init-core-data-001
   [junit4]   2> 2815921 T4107 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /_/
   [junit4]   2> 2815927 T4107 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 2815928 T4107 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2815929 T4108 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 2816030 T4107 oasc.ZkTestServer.run start zk server on port:52399
   [junit4]   2> 2816032 T4107 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2816036 T4114 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@74c6ed97 name:ZooKeeperConnection Watcher:127.0.0.1:52399 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2816036 T4107 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2816036 T4107 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 2816047 T4107 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2816049 T4116 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@25883166 name:ZooKeeperConnection Watcher:127.0.0.1:52399/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2816049 T4107 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2816049 T4107 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 2816059 T4107 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 2816062 T4107 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 2816064 T4107 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 2816073 T4107 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2816074 T4107 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2816086 T4107 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2816087 T4107 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 2816192 T4107 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2816193 T4107 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2816196 T4107 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2816196 T4107 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2816199 T4107 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2816200 T4107 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2816207 T4107 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2816208 T4107 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 2816211 T4107 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2816211 T4107 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2816214 T4107 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2816215 T4107 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2816223 T4107 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2816223 T4107 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2816226 T4107 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2816227 T4107 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2816230 T4107 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2816230 T4107 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2817318 T4107 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2817323 T4107 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:52403
   [junit4]   2> 2817323 T4107 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2817323 T4107 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2817324 T4107 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-002
   [junit4]   2> 2817324 T4107 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-002/'
   [junit4]   2> 2817357 T4107 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-002/solr.xml
   [junit4]   2> 2817438 T4107 oasc.CoreContainer.<init> New CoreContainer 1848427469
   [junit4]   2> 2817438 T4107 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-002/]
   [junit4]   2> 2817439 T4107 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2817439 T4107 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2817439 T4107 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2817440 T4107 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2817440 T4107 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2817440 T4107 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2817440 T4107 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2817441 T4107 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2817441 T4107 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2817441 T4107 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2817442 T4107 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2817442 T4107 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2817442 T4107 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52399/solr
   [junit4]   2> 2817443 T4107 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2817443 T4107 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2817446 T4127 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3ceb003d name:ZooKeeperConnection Watcher:127.0.0.1:52399 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2817447 T4107 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2817461 T4107 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2817463 T4129 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@77d4e3c0 name:ZooKeeperConnection Watcher:127.0.0.1:52399/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2817463 T4107 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2817465 T4107 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 2817469 T4107 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 2817472 T4107 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 2817475 T4107 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 2817478 T4107 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 2817481 T4107 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 2817483 T4107 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52402__
   [junit4]   2> 2817484 T4107 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52402__
   [junit4]   2> 2817487 T4107 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 2817489 T4107 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 2817500 T4107 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:52402__
   [junit4]   2> 2817501 T4107 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 2817503 T4107 oasc.Overseer.start Overseer (id=92040992474202115-127.0.0.1:52402__-n_0000000000) starting
   [junit4]   2> 2817513 T4107 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 2817538 T4131 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 2817538 T4107 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 2817548 T4107 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 2817549 T4107 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2817551 T4130 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 2817555 T4132 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 2817555 T4132 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2817556 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2817556 T4132 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2817557 T4130 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2817558 T4130 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52402/_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52402__",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 2817558 T4130 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 2817558 T4130 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 2817561 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2817570 T4129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2818558 T4132 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2818559 T4132 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 2818560 T4132 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2818560 T4132 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 2818561 T4132 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2818561 T4132 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-002/collection1/'
   [junit4]   2> 2818562 T4132 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 2818563 T4132 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 2818563 T4132 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 2818630 T4132 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
   [junit4]   2> 2818678 T4132 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2818780 T4132 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2818793 T4132 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 2819222 T4132 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 2819223 T4132 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 2819224 T4132 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2819237 T4132 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2819240 T4132 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2819252 T4132 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2819257 T4132 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2819261 T4132 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2819261 T4132 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2819262 T4132 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2819262 T4132 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2819263 T4132 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2819263 T4132 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2819264 T4132 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2819264 T4132 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2819264 T4132 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-002/collection1/, dataDir=./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/control/data/
   [junit4]   2> 2819265 T4132 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@299a3d4
   [junit4]   2> 2819265 T4132 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/control/data
   [junit4]   2> 2819266 T4132 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/control/data/index/
   [junit4]   2> 2819266 T4132 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2819266 T4132 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/control/data/index
   [junit4]   2> 2819267 T4132 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=84.078125, floorSegmentMB=0.9296875, forceMergeDeletesPctAllowed=17.506918862156883, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2819268 T4132 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6fe6e00e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@37cd0f19),segFN=segments_1,generation=1}
   [junit4]   2> 2819268 T4132 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2819272 T4132 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2819273 T4132 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2819273 T4132 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2819273 T4132 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2819274 T4132 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2819274 T4132 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2819275 T4132 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2819275 T4132 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2819275 T4132 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2819276 T4132 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2819276 T4132 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2819277 T4132 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2819277 T4132 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2819277 T4132 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2819278 T4132 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2819278 T4132 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2819297 T4132 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2819303 T4132 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2819303 T4132 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2819304 T4132 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=35, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=98.578125, floorSegmentMB=1.853515625, forceMergeDeletesPctAllowed=9.9755902801654, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2819304 T4132 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@6fe6e00e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@37cd0f19),segFN=segments_1,generation=1}
   [junit4]   2> 2819305 T4132 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2819305 T4132 oass.SolrIndexSearcher.<init> Opening Searcher@227d51bc[collection1] main
   [junit4]   2> 2819305 T4132 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 2819306 T4132 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2819306 T4132 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2819307 T4132 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2819307 T4132 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 2819307 T4132 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2819308 T4132 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2819308 T4132 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2819309 T4132 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 2819309 T4132 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 2819312 T4133 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@227d51bc[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2819314 T4132 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2819315 T4136 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52402/_ collection:control_collection shard:shard1
   [junit4]   2> 2819315 T4107 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 2819316 T4107 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2819316 T4136 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2819317 T4107 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2819320 T4139 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@523cc68c name:ZooKeeperConnection Watcher:127.0.0.1:52399/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2819321 T4107 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2819334 T4107 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2819346 T4136 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 2819346 T4107 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2819348 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2819348 T4136 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 2819349 T4136 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C712 name=collection1 org.apache.solr.core.SolrCore@313216db url=http://127.0.0.1:52402/_/collection1 node=127.0.0.1:52402__ C712_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:52402/_, core=collection1, node_name=127.0.0.1:52402__}
   [junit4]   2> 2819349 T4136 C712 P52402 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52402/_/collection1/
   [junit4]   2> 2819349 T4136 C712 P52402 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 2819349 T4130 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2819349 T4136 C712 P52402 oasc.SyncStrategy.syncToMe http://127.0.0.1:52402/_/collection1/ has no replicas
   [junit4]   2> 2819350 T4136 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52402/_/collection1/ shard1
   [junit4]   2> 2819350 T4136 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2819359 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2819367 T4139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2819367 T4129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2819380 T4130 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2819383 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2819486 T4139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2819486 T4129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2819532 T4136 oasc.ZkController.register We are http://127.0.0.1:52402/_/collection1/ and leader is http://127.0.0.1:52402/_/collection1/
   [junit4]   2> 2819533 T4136 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52402/_
   [junit4]   2> 2819533 T4136 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 2819533 T4136 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 2819533 T4136 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2819535 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2819535 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2819535 T4136 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2819535 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2819537 T4130 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2819537 T4130 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52402/_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52402__",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 2819540 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2819643 T4129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2819643 T4139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 2820388 T4107 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 2820389 T4107 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2820391 T4107 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:52408
   [junit4]   2> 2820392 T4107 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2820392 T4107 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2820392 T4107 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-003
   [junit4]   2> 2820393 T4107 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-003/'
   [junit4]   2> 2820426 T4107 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-003/solr.xml
   [junit4]   2> 2820501 T4107 oasc.CoreContainer.<init> New CoreContainer 880926921
   [junit4]   2> 2820502 T4107 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-003/]
   [junit4]   2> 2820503 T4107 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2820503 T4107 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2820503 T4107 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2820503 T4107 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2820503 T4107 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2820504 T4107 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2820504 T4107 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2820504 T4107 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2820504 T4107 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2820505 T4107 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2820505 T4107 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2820506 T4107 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2820506 T4107 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52399/solr
   [junit4]   2> 2820506 T4107 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2820507 T4107 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2820533 T4150 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@3accc70d name:ZooKeeperConnection Watcher:127.0.0.1:52399 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2820533 T4107 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2820549 T4107 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2820551 T4152 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@736bc79b name:ZooKeeperConnection Watcher:127.0.0.1:52399/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2820551 T4107 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2820576 T4107 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2821579 T4107 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52407__
   [junit4]   2> 2821581 T4107 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52407__
   [junit4]   2> 2821585 T4139 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 2821585 T4152 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 2821585 T4129 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 2821600 T4153 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 2821600 T4153 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2821601 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2821601 T4153 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2821602 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2821602 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2821603 T4130 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2821603 T4130 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52407/_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52407__",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 2821604 T4130 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 2821604 T4130 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 2821607 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2821617 T4139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2821617 T4129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2821617 T4152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2822603 T4153 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2822604 T4153 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 2822604 T4153 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2822605 T4153 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 2822606 T4153 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2822606 T4153 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-003/collection1/'
   [junit4]   2> 2822607 T4153 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 2822608 T4153 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 2822608 T4153 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 2822675 T4153 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
   [junit4]   2> 2822723 T4153 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2822824 T4153 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2822835 T4153 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 2823262 T4153 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 2823264 T4153 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 2823264 T4153 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2823277 T4153 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2823280 T4153 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2823292 T4153 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2823296 T4153 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2823300 T4153 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2823301 T4153 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2823302 T4153 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2823302 T4153 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2823303 T4153 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2823303 T4153 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2823303 T4153 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2823304 T4153 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2823304 T4153 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-003/collection1/, dataDir=./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/jetty1/
   [junit4]   2> 2823304 T4153 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@299a3d4
   [junit4]   2> 2823305 T4153 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/jetty1
   [junit4]   2> 2823305 T4153 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/jetty1/index/
   [junit4]   2> 2823306 T4153 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 2823306 T4153 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/jetty1/index
   [junit4]   2> 2823306 T4153 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=84.078125, floorSegmentMB=0.9296875, forceMergeDeletesPctAllowed=17.506918862156883, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2823307 T4153 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@46875b26 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@576e8454),segFN=segments_1,generation=1}
   [junit4]   2> 2823308 T4153 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2823313 T4153 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2823314 T4153 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2823314 T4153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2823314 T4153 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2823315 T4153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2823315 T4153 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2823316 T4153 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2823316 T4153 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2823316 T4153 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2823317 T4153 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2823317 T4153 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2823318 T4153 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2823318 T4153 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2823319 T4153 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2823319 T4153 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2823320 T4153 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2823339 T4153 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2823343 T4153 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2823343 T4153 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2823344 T4153 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=35, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=98.578125, floorSegmentMB=1.853515625, forceMergeDeletesPctAllowed=9.9755902801654, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2823345 T4153 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@46875b26 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@576e8454),segFN=segments_1,generation=1}
   [junit4]   2> 2823345 T4153 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2823345 T4153 oass.SolrIndexSearcher.<init> Opening Searcher@214c6b9a[collection1] main
   [junit4]   2> 2823346 T4153 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 2823347 T4153 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2823347 T4153 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2823347 T4153 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2823348 T4153 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 2823348 T4153 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2823348 T4153 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2823349 T4153 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2823349 T4153 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 2823349 T4153 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 2823354 T4154 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@214c6b9a[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2823356 T4153 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2823357 T4157 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52407/_ collection:collection1 shard:shard1
   [junit4]   2> 2823358 T4107 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 2823358 T4107 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2823358 T4157 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 2823375 T4157 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 2823378 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2823378 T4157 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 2823378 T4157 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C713 name=collection1 org.apache.solr.core.SolrCore@471bc16a url=http://127.0.0.1:52407/_/collection1 node=127.0.0.1:52407__ C713_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:52407/_, core=collection1, node_name=127.0.0.1:52407__}
   [junit4]   2> 2823378 T4157 C713 P52407 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52407/_/collection1/
   [junit4]   2> 2823379 T4157 C713 P52407 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 2823379 T4130 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2823379 T4157 C713 P52407 oasc.SyncStrategy.syncToMe http://127.0.0.1:52407/_/collection1/ has no replicas
   [junit4]   2> 2823379 T4157 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52407/_/collection1/ shard1
   [junit4]   2> 2823380 T4157 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 2823382 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2823383 T4152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2823384 T4139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2823383 T4129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2823406 T4130 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2823409 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2823512 T4139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2823512 T4129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2823512 T4152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2823559 T4157 oasc.ZkController.register We are http://127.0.0.1:52407/_/collection1/ and leader is http://127.0.0.1:52407/_/collection1/
   [junit4]   2> 2823560 T4157 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52407/_
   [junit4]   2> 2823560 T4157 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 2823560 T4157 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 2823560 T4157 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2823562 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2823562 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2823562 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2823562 T4157 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2823563 T4130 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2823564 T4130 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52407/_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52407__",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 2823573 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2823676 T4139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2823676 T4152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2823676 T4129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 2824394 T4107 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 2824395 T4107 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 2824397 T4107 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:52412
   [junit4]   2> 2824398 T4107 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 2824398 T4107 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2824398 T4107 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-004
   [junit4]   2> 2824399 T4107 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-004/'
   [junit4]   2> 2824432 T4107 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-004/solr.xml
   [junit4]   2> 2824507 T4107 oasc.CoreContainer.<init> New CoreContainer 2119335915
   [junit4]   2> 2824508 T4107 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-004/]
   [junit4]   2> 2824509 T4107 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 2824509 T4107 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 2824509 T4107 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 2824509 T4107 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 2824510 T4107 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 2824510 T4107 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 2824510 T4107 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 2824510 T4107 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 2824511 T4107 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 2824511 T4107 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2824511 T4107 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2824512 T4107 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 2824512 T4107 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52399/solr
   [junit4]   2> 2824512 T4107 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 2824513 T4107 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2824527 T4169 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5a4c5e17 name:ZooKeeperConnection Watcher:127.0.0.1:52399 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2824527 T4107 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2824542 T4107 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 2824544 T4171 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6e6eb68e name:ZooKeeperConnection Watcher:127.0.0.1:52399/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2824544 T4107 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 2824569 T4107 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 2825573 T4107 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52411__
   [junit4]   2> 2825575 T4107 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:52411__
   [junit4]   2> 2825579 T4152 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 2825579 T4171 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 2825579 T4139 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 2825579 T4129 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 2825597 T4172 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 2825597 T4172 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2825599 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2825599 T4172 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 2825599 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2825599 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2825600 T4130 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2825600 T4130 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52411/_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52411__",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 2825601 T4130 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 2825601 T4130 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 2825604 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2825605 T4171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2825605 T4152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2825605 T4129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2825605 T4139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2826600 T4172 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2826601 T4172 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 2826601 T4172 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 2826602 T4172 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 2826602 T4172 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2826603 T4172 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-004/collection1/'
   [junit4]   2> 2826604 T4172 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-004/collection1/lib/classes/' to classloader
   [junit4]   2> 2826605 T4172 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-004/collection1/lib/README' to classloader
   [junit4]   2> 2826605 T4172 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-004/collection1/lib/.svn/' to classloader
   [junit4]   2> 2826671 T4172 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
   [junit4]   2> 2826719 T4172 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2826821 T4172 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 2826832 T4172 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 2827258 T4172 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 2827259 T4172 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 2827260 T4172 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2827273 T4172 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 2827276 T4172 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 2827288 T4172 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2827292 T4172 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 2827296 T4172 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2827297 T4172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2827297 T4172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2827298 T4172 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2827299 T4172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 2827299 T4172 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 2827299 T4172 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 2827299 T4172 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2827300 T4172 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-004/collection1/, dataDir=./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/jetty2/
   [junit4]   2> 2827300 T4172 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@299a3d4
   [junit4]   2> 2827301 T4172 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/jetty2
   [junit4]   2> 2827301 T4172 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/jetty2/index/
   [junit4]   2> 2827301 T4172 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 2827302 T4172 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.HttpPartitionTest-84A7796AF29B22FB-001/tempDir-001/jetty2/index
   [junit4]   2> 2827302 T4172 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=84.078125, floorSegmentMB=0.9296875, forceMergeDeletesPctAllowed=17.506918862156883, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2827303 T4172 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@71528b6e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@45e89ddb),segFN=segments_1,generation=1}
   [junit4]   2> 2827303 T4172 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2827309 T4172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2827310 T4172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2827310 T4172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2827310 T4172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2827310 T4172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2827311 T4172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2827311 T4172 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2827312 T4172 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2827312 T4172 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2827313 T4172 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 2827313 T4172 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 2827313 T4172 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 2827314 T4172 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 2827314 T4172 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 2827315 T4172 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 2827316 T4172 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 2827334 T4172 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 2827339 T4172 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 2827339 T4172 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 2827340 T4172 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=35, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=98.578125, floorSegmentMB=1.853515625, forceMergeDeletesPctAllowed=9.9755902801654, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 2827340 T4172 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@71528b6e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@45e89ddb),segFN=segments_1,generation=1}
   [junit4]   2> 2827341 T4172 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 2827341 T4172 oass.SolrIndexSearcher.<init> Opening Searcher@18a4dc0b[collection1] main
   [junit4]   2> 2827341 T4172 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 2827342 T4172 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 2827342 T4172 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2827343 T4172 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2827343 T4172 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 2827343 T4172 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2827344 T4172 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2827344 T4172 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2827345 T4172 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 2827345 T4172 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 2827349 T4173 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@18a4dc0b[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 2827352 T4172 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 2827352 T4176 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:52411/_ collection:collection1 shard:shard2
   [junit4]   2> 2827353 T4107 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 2827353 T4107 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 2827353 T4176 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 2827355 T4107 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2827356 T4107 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 30000 for each attempt
   [junit4]   2> 2827356 T4107 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30000
   [junit4]   2> 2827357 T4107 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2827370 T4176 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 2827379 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2827380 T4176 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 2827380 T4176 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C714 name=collection1 org.apache.solr.core.SolrCore@5413c6d8 url=http://127.0.0.1:52411/_/collection1 node=127.0.0.1:52411__ C714_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:52411/_, core=collection1, node_name=127.0.0.1:52411__}
   [junit4]   2> 2827380 T4176 C714 P52411 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:52411/_/collection1/
   [junit4]   2> 2827380 T4130 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2827380 T4176 C714 P52411 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 2827381 T4176 C714 P52411 oasc.SyncStrategy.syncToMe http://127.0.0.1:52411/_/collection1/ has no replicas
   [junit4]   2> 2827381 T4176 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:52411/_/collection1/ shard2
   [junit4]   2> 2827381 T4176 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 2827383 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2827384 T4129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2827384 T4139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2827384 T4171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2827384 T4152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2827388 T4130 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2827391 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2827494 T4129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2827494 T4171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2827494 T4139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2827494 T4152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2827540 T4176 oasc.ZkController.register We are http://127.0.0.1:52411/_/collection1/ and leader is http://127.0.0.1:52411/_/collection1/
   [junit4]   2> 2827541 T4176 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52411/_
   [junit4]   2> 2827541 T4176 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 2827541 T4176 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 2827541 T4176 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 2827543 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2827543 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2827543 T4176 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2827543 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2827544 T4130 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2827545 T4130 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:52411/_",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:52411__",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 2827554 T4129 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 2827657 T4129 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2827657 T4171 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2827657 T4152 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2827657 T4139 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 2828360 T4107 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 2828361 T4107 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
   [junit4]   2> 2828362 T4137 oasc.SocketProxy$Acceptor.run accepted Socket[addr=/140.211.11.196,port=52415,localport=52402], receiveBufferSize:6565888
   [junit4]   2> 2828363 T4137 oasc.SocketProxy$Bridge.<init> proxy connection Socket[addr=/127.0.0.1,port=52403,localport=52416], receiveBufferSize=6565888
   [junit4]   2> ASYNC  NEW_CORE C715 name=collection1 org.apache.solr.core.SolrCore@313216db url=http://127.0.0.1:52402/_/collection1 node=127.0.0.1:52402__ C715_STATE=coll:control_collection core:collection1 props:{state=active, base_url=http://127.0.0.1:52402/_, core=collection1, node_name=127.0.0.1:52402__, leader=true}
   [junit4]   2> 2828364 T4119 C715 P52402 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2828366 T4119 C715 P52402 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2828367 T4119 C715 P52402 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 2828367 T4119 C715 P52402 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 2828368 T4119 C715 P52402 oasup.LogUpdateProcessor.finish [collection1] webapp=/_ path=/update params={softCommit=false&commit=true&wt=javabin&waitSearcher=true&version=2} {commit=} 0 4
   [junit4]   2> 2828389 T4158 oasc.SocketProxy$Acceptor.run accepted Socket[addr=/140.211.11.196,port=52417,localport=52407], receiveBufferSize:6565888
   [junit4]   2> 2828390 T4158 oasc.SocketProxy$Bridge.<init> proxy connection Socket[addr=/127.0.0.1,port=52408,localport=52418], receiveBufferSize=6565888
   [junit4]   2> 2828393 T4158 oasc.SocketProxy$Acceptor.run accepted Socket[addr=/140.211.11.196,port=52419,localport=52407], receiveBufferSize:6565888
   [junit4]   2> 2828393 T4158 oasc.SocketProxy$Bridge.<init> proxy connection Socket[addr=/127.0.0.1,port=52408,localport=52420], receiveBufferSize=6565888
   [junit4]   2> ASYNC  NEW_CORE C716 name=collection1 org.apache.solr.core.SolrCore@471bc16a url=http://127.0.0.1:52407/_/collection1 node=127.0.0.1:52407__ C716_STATE=coll:collection1 core:collection1 props:{state=active, base_url=http://127.0.0.1:52407/_, core=collection1, node_name=127.0.0.1:52407__, leader=true}
   [junit4]   2> 2828394 T4143 C716 P52407 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2828394 T4143 C716 P52407 oasu.DirectUpdateHandler2.commit No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 2828395 T4143 C716 P52407 oasc.SolrCore.openNewSearcher SolrIndexSearcher has not changed - not re-opening: org.apache.solr.search.SolrIndexSearcher
   [junit4]   2> 2828395 T4143 C716 P52407 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 2828396 T4143 C716 P52407 oasup.LogUpdateProcessor.finish [collection1] webapp=/_ path=/update params={softCommit=false&distrib.from=http://127.0.0.1:52407/_/collection1/&commit=true&commit_end_point=true&wt=javabin&expungeDeletes=false&update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&version=2} {commit=} 0 2
   [junit4]   2> 2828497 T4177 oasc.SocketProxy$Acceptor.run accepted Socket[addr=/140.211.11.196,port=52421,localport=52411], receiveBufferSize:6565888
   [junit4]   2> 2828498 T4177 oasc.SocketProxy$Bridge.<init> proxy connection Socket[addr=/127.0.0.1,port=52412,localport=52422], receiveBufferSize=6565888
   [junit4]   2> ASYNC  NEW_CORE C717 name=collection1 org.apache.solr.core.SolrCore@5413c6d8 url=http://127.0.0.1:52411/_/collectio

[...truncated too long message...]

.java:505)
   [junit4]   2> 	   5) Thread[id=1020, name=coreZkRegister-614-thread-1, state=WAITING, group=TGRP-CollectionsAPIAsyncDistributedZkTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   6) Thread[id=1017, name=searcherExecutor-619-thread-1, state=WAITING, group=TGRP-CollectionsAPIAsyncDistributedZkTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   7) Thread[id=1032, name=parallelCoreAdminExecutor-617-thread-1, state=WAITING, group=TGRP-CollectionsAPIAsyncDistributedZkTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   8) Thread[id=1146, name=searcherExecutor-643-thread-1, state=WAITING, group=TGRP-CollectionsAPIAsyncDistributedZkTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   9) Thread[id=1141, name=parallelCoreAdminExecutor-617-thread-2, state=WAITING, group=TGRP-CollectionsAPIAsyncDistributedZkTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]   2> 	        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]   2> 	        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	  10) Thread[id=1010, name=TEST-CollectionsAPIAsyncDistributedZkTest.testDistribSearch-seed#[84A7796AF29B22FB]-SendThread(127.0.0.1:41431), state=RUNNABLE, group=TGRP-CollectionsAPIAsyncDistributedZkTest]
   [junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(Native Method)
   [junit4]   2> 	        at sun.nio.ch.KQueueArrayWrapper.interrupt(KQueueArrayWrapper.java:205)
   [junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.wakeup(KQueueSelectorImpl.java:243)
   [junit4]   2> 	        at java.nio.channels.spi.AbstractSelector$1.interrupt(AbstractSelector.java:210)
   [junit4]   2> 	        at java.nio.channels.spi.AbstractSelector.begin(AbstractSelector.java:216)
   [junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:102)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> 	  11) Thread[id=940, name=TEST-CollectionsAPIAsyncDistributedZkTest.testDistribSearch-seed#[84A7796AF29B22FB]-SendThread(127.0.0.1:41431), state=RUNNABLE, group=TGRP-CollectionsAPIAsyncDistributedZkTest]
   [junit4]   2> 	        at sun.nio.ch.IOUtil.drain(Native Method)
   [junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.updateSelectedKeys(KQueueSelectorImpl.java:167)
   [junit4]   2> 	        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:108)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
   [junit4]   2> 	        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)
   [junit4]   2> 	        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
   [junit4]   2> NOTE: test params are: codec=Lucene45, sim=DefaultSimilarity, locale=it_IT, timezone=Asia/Dubai
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=13,free=267752352,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [CustomCollectionTest, ChangedSchemaMergeTest, TestFuzzyAnalyzedSuggestions, SolrCmdDistributorTest, SimplePostToolTest, TestSolrXmlPersistor, SuggesterFSTTest, BadComponentTest, TestReplicationHandlerBackup, TestFieldTypeCollectionResource, HdfsBasicDistributedZk2Test, TestSystemIdResolver, SliceStateTest, TestSolrXml, FileUtilsTest, ResponseLogComponentTest, TestPseudoReturnFields, OpenExchangeRatesOrgProviderTest, TestSuggestSpellingConverter, SimpleFacetsTest, ShardSplitTest, RAMDirectoryFactoryTest, TestLMDirichletSimilarityFactory, BadIndexSchemaTest, DocValuesMultiTest, TestBadConfig, PrimitiveFieldTypeTest, SolrCoreCheckLockOnStartupTest, CollectionsAPIAsyncDistributedZkTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIAsyncDistributedZkTest -Dtests.seed=84A7796AF29B22FB -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=it_IT -Dtests.timezone=Asia/Dubai -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s J1 | CollectionsAPIAsyncDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([84A7796AF29B22FB]:0)
   [junit4] Completed on J1 in 7223.81s, 1 test, 2 errors <<< FAILURES!

[...truncated 230 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1299: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:923: There were test failures: 404 suites, 1654 tests, 1 suite-level error, 2 errors, 333 ignored (13 assumptions)

Total time: 169 minutes 41 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-4.x-Java7 #2008
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 137431407 bytes
Compression is 0.0%
Took 24 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2020 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2020/

All tests passed

Build Log:
[...truncated 29247 lines...]
check-licenses:
     [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/log4j-1.2.16.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/log4j-1.2.16.jar.sha1

[...truncated 1 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:70: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.

Total time: 139 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-4.x-Java7 #2008
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 19 ms
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2019 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2019/

1 tests failed.
REGRESSION:  org.apache.solr.cloud.HttpPartitionTest.testDistribSearch

Error Message:
Didn't see all replicas come up within 30000 ms! ClusterState: live nodes:[127.0.0.1:44776_, 127.0.0.1:16261_, 127.0.0.1:12802_] collections:{control_collection=DocCollection(control_collection)={   "replicationFactor":"1",   "maxShardsPerNode":"1",   "router":{"name":"compositeId"},   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{"core_node1":{           "state":"active",           "base_url":"http://127.0.0.1:12802",           "core":"collection1",           "node_name":"127.0.0.1:12802_",           "leader":"true"}}}},   "autoCreated":"true"}, collection1=DocCollection(collection1)={   "replicationFactor":"1",   "maxShardsPerNode":"1",   "router":{"name":"compositeId"},   "shards":{     "shard1":{       "range":"80000000-ffffffff",       "state":"active",       "replicas":{"core_node2":{           "state":"active",           "base_url":"http://127.0.0.1:44776",           "core":"collection1",           "node_name":"127.0.0.1:44776_",           "leader":"true"}}},     "shard2":{       "range":"0-7fffffff",       "state":"active",       "replicas":{"core_node1":{           "state":"active",           "base_url":"http://127.0.0.1:16261",           "core":"collection1",           "node_name":"127.0.0.1:16261_",           "leader":"true"}}}},   "autoCreated":"true"}, c8n_1x2=DocCollection(c8n_1x2)={   "replicationFactor":"2",   "maxShardsPerNode":"1",   "router":{"name":"compositeId"},   "shards":{"shard1":{       "range":"80000000-7fffffff",       "state":"active",       "replicas":{         "core_node1":{           "state":"recovering",           "base_url":"http://127.0.0.1:44776",           "core":"c8n_1x2_shard1_replica1",           "node_name":"127.0.0.1:44776_"},         "core_node2":{           "state":"active",           "base_url":"http://127.0.0.1:12802",           "core":"c8n_1x2_shard1_replica2",           "node_name":"127.0.0.1:12802_",           "leader":"true"}}}}}}

Stack Trace:
java.lang.AssertionError: Didn't see all replicas come up within 30000 ms! ClusterState: live nodes:[127.0.0.1:44776_, 127.0.0.1:16261_, 127.0.0.1:12802_] collections:{control_collection=DocCollection(control_collection)={
  "replicationFactor":"1",
  "maxShardsPerNode":"1",
  "router":{"name":"compositeId"},
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{"core_node1":{
          "state":"active",
          "base_url":"http://127.0.0.1:12802",
          "core":"collection1",
          "node_name":"127.0.0.1:12802_",
          "leader":"true"}}}},
  "autoCreated":"true"}, collection1=DocCollection(collection1)={
  "replicationFactor":"1",
  "maxShardsPerNode":"1",
  "router":{"name":"compositeId"},
  "shards":{
    "shard1":{
      "range":"80000000-ffffffff",
      "state":"active",
      "replicas":{"core_node2":{
          "state":"active",
          "base_url":"http://127.0.0.1:44776",
          "core":"collection1",
          "node_name":"127.0.0.1:44776_",
          "leader":"true"}}},
    "shard2":{
      "range":"0-7fffffff",
      "state":"active",
      "replicas":{"core_node1":{
          "state":"active",
          "base_url":"http://127.0.0.1:16261",
          "core":"collection1",
          "node_name":"127.0.0.1:16261_",
          "leader":"true"}}}},
  "autoCreated":"true"}, c8n_1x2=DocCollection(c8n_1x2)={
  "replicationFactor":"2",
  "maxShardsPerNode":"1",
  "router":{"name":"compositeId"},
  "shards":{"shard1":{
      "range":"80000000-7fffffff",
      "state":"active",
      "replicas":{
        "core_node1":{
          "state":"recovering",
          "base_url":"http://127.0.0.1:44776",
          "core":"c8n_1x2_shard1_replica1",
          "node_name":"127.0.0.1:44776_"},
        "core_node2":{
          "state":"active",
          "base_url":"http://127.0.0.1:12802",
          "core":"c8n_1x2_shard1_replica2",
          "node_name":"127.0.0.1:12802_",
          "leader":"true"}}}}}}
	at __randomizedtesting.SeedInfo.seed([2F6FD45E5446A9CF:AE895A462319C9F3]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.HttpPartitionTest.ensureAllReplicasAreActive(HttpPartitionTest.java:505)
	at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:215)
	at org.apache.solr.cloud.HttpPartitionTest.doTest(HttpPartitionTest.java:142)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11015 lines...]
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4]   2> Creating dataDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/init-core-data-001
   [junit4]   2> 1185260 T2941 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 1185265 T2941 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 1185266 T2941 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1185267 T2942 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1185367 T2941 oasc.ZkTestServer.run start zk server on port:15777
   [junit4]   2> 1185368 T2941 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1185374 T2948 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5b101d3f name:ZooKeeperConnection Watcher:127.0.0.1:15777 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1185374 T2941 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1185374 T2941 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1185390 T2941 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1185392 T2950 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f8a98f name:ZooKeeperConnection Watcher:127.0.0.1:15777/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1185392 T2941 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1185392 T2941 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 1185402 T2941 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 1185406 T2941 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 1185408 T2941 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 1185416 T2941 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1185417 T2941 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1185430 T2941 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1185430 T2941 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 1185435 T2941 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1185436 T2941 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1185438 T2941 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1185439 T2941 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1185442 T2941 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1185443 T2941 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1185451 T2941 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1185452 T2941 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 1185455 T2941 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1185455 T2941 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1185458 T2941 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1185459 T2941 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1185462 T2941 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1185462 T2941 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1185465 T2941 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1185466 T2941 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1185474 T2941 oasc.AbstractZkTestCase.putConfig put /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1185474 T2941 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1186447 T2941 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1186451 T2941 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:36008
   [junit4]   2> 1186451 T2941 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1186452 T2941 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1186452 T2941 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-002
   [junit4]   2> 1186452 T2941 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-002/'
   [junit4]   2> 1186485 T2941 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-002/solr.xml
   [junit4]   2> 1186539 T2941 oasc.CoreContainer.<init> New CoreContainer 736962122
   [junit4]   2> 1186539 T2941 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-002/]
   [junit4]   2> 1186540 T2941 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1186540 T2941 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1186541 T2941 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1186541 T2941 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1186541 T2941 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1186541 T2941 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1186542 T2941 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1186542 T2941 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1186542 T2941 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1186543 T2941 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1186543 T2941 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1186543 T2941 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1186544 T2941 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:15777/solr
   [junit4]   2> 1186544 T2941 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1186545 T2941 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1186547 T2961 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e6cdacd name:ZooKeeperConnection Watcher:127.0.0.1:15777 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1186548 T2941 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1186564 T2941 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1186565 T2963 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@615ff8c1 name:ZooKeeperConnection Watcher:127.0.0.1:15777/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1186566 T2941 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1186569 T2941 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1186573 T2941 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 1186584 T2941 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 1186596 T2941 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 1186606 T2941 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 1186609 T2941 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 1186611 T2941 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:12802_
   [junit4]   2> 1186619 T2941 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:12802_
   [junit4]   2> 1186622 T2941 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 1186624 T2941 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 1186638 T2941 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:12802_
   [junit4]   2> 1186638 T2941 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1186641 T2941 oasc.Overseer.start Overseer (id=92034725069455363-127.0.0.1:12802_-n_0000000000) starting
   [junit4]   2> 1186650 T2941 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 1186668 T2965 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1186668 T2941 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 1186670 T2941 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 1186672 T2941 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1186674 T2964 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1186679 T2966 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 1186679 T2966 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1186680 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1186680 T2966 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1186681 T2964 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1186682 T2964 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=1 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:12802",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:12802_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1186682 T2964 oasc.Overseer$ClusterStateUpdater.createCollection Create collection control_collection with shards [shard1]
   [junit4]   2> 1186683 T2964 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1186685 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1186686 T2963 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1187681 T2966 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1187682 T2966 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 1187683 T2966 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1187683 T2966 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 1187684 T2966 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1187684 T2966 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-002/collection1/'
   [junit4]   2> 1187685 T2966 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 1187686 T2966 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 1187687 T2966 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 1187754 T2966 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
   [junit4]   2> 1187803 T2966 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1187805 T2966 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1187817 T2966 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1188281 T2966 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1188282 T2966 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1188283 T2966 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1188297 T2966 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1188300 T2966 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1188313 T2966 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1188318 T2966 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1188322 T2966 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1188323 T2966 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1188323 T2966 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1188323 T2966 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1188324 T2966 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1188325 T2966 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1188325 T2966 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1188325 T2966 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1188325 T2966 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-002/collection1/, dataDir=./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/control/data/
   [junit4]   2> 1188326 T2966 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@50fe766d
   [junit4]   2> 1188326 T2966 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/control/data
   [junit4]   2> 1188327 T2966 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/control/data/index/
   [junit4]   2> 1188327 T2966 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1188327 T2966 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/control/data/index
   [junit4]   2> 1188328 T2966 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=91.1650390625, floorSegmentMB=0.2197265625, forceMergeDeletesPctAllowed=7.796980668834749, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1188329 T2966 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@5b162cb9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@38ea8b10),segFN=segments_1,generation=1}
   [junit4]   2> 1188329 T2966 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1188333 T2966 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1188334 T2966 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1188334 T2966 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1188335 T2966 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1188335 T2966 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1188335 T2966 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1188336 T2966 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1188336 T2966 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1188336 T2966 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1188337 T2966 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1188337 T2966 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1188338 T2966 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1188338 T2966 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1188338 T2966 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1188339 T2966 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1188339 T2966 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1188358 T2966 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1188364 T2966 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1188364 T2966 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1188365 T2966 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=50, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7941882430337752]
   [junit4]   2> 1188365 T2966 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@5b162cb9 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@38ea8b10),segFN=segments_1,generation=1}
   [junit4]   2> 1188366 T2966 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1188366 T2966 oass.SolrIndexSearcher.<init> Opening Searcher@391276c0[collection1] main
   [junit4]   2> 1188366 T2966 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 1188367 T2966 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1188367 T2966 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1188368 T2966 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1188368 T2966 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1188368 T2966 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1188369 T2966 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1188369 T2966 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1188370 T2966 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 1188370 T2966 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1188373 T2967 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@391276c0[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1188375 T2966 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1188376 T2970 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:12802 collection:control_collection shard:shard1
   [junit4]   2> 1188376 T2941 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
   [junit4]   2> 1188377 T2941 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1188377 T2970 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1188381 T2941 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1188393 T2973 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@198ca369 name:ZooKeeperConnection Watcher:127.0.0.1:15777/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1188394 T2941 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1188395 T2941 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1188396 T2970 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 1188398 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1188398 T2941 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1188398 T2970 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1188399 T2970 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C911 name=collection1 org.apache.solr.core.SolrCore@3994bee8 url=http://127.0.0.1:12802/collection1 node=127.0.0.1:12802_ C911_STATE=coll:control_collection core:collection1 props:{state=down, base_url=http://127.0.0.1:12802, core=collection1, node_name=127.0.0.1:12802_}
   [junit4]   2> 1188399 T2970 C911 P12802 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:12802/collection1/
   [junit4]   2> 1188399 T2970 C911 P12802 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1188399 T2964 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1188399 T2970 C911 P12802 oasc.SyncStrategy.syncToMe http://127.0.0.1:12802/collection1/ has no replicas
   [junit4]   2> 1188400 T2970 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:12802/collection1/ shard1
   [junit4]   2> 1188400 T2970 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1188402 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1188403 T2963 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1188403 T2973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1188408 T2964 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1188411 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1188513 T2963 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1188513 T2973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1188560 T2970 oasc.ZkController.register We are http://127.0.0.1:12802/collection1/ and leader is http://127.0.0.1:12802/collection1/
   [junit4]   2> 1188561 T2970 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:12802
   [junit4]   2> 1188561 T2970 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1188561 T2970 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 1188561 T2970 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1188563 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1188563 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1188563 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1188563 T2970 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1188564 T2964 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1188565 T2964 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:12802",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:12802_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 1188575 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1188696 T2973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1188696 T2963 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1189400 T2941 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 1189401 T2941 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1189403 T2941 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:51168
   [junit4]   2> 1189405 T2941 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1189405 T2941 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1189405 T2941 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-003
   [junit4]   2> 1189405 T2941 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-003/'
   [junit4]   2> 1189438 T2941 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-003/solr.xml
   [junit4]   2> 1189492 T2941 oasc.CoreContainer.<init> New CoreContainer 2065350055
   [junit4]   2> 1189492 T2941 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-003/]
   [junit4]   2> 1189493 T2941 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1189493 T2941 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1189494 T2941 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1189494 T2941 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1189494 T2941 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1189494 T2941 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1189495 T2941 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1189495 T2941 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1189495 T2941 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1189496 T2941 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1189496 T2941 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1189496 T2941 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1189496 T2941 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:15777/solr
   [junit4]   2> 1189497 T2941 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1189497 T2941 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1189525 T2984 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@18995e15 name:ZooKeeperConnection Watcher:127.0.0.1:15777 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1189526 T2941 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1189529 T2941 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1189540 T2986 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@308ed264 name:ZooKeeperConnection Watcher:127.0.0.1:15777/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1189540 T2941 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1189556 T2941 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1190559 T2941 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:16261_
   [junit4]   2> 1190561 T2941 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:16261_
   [junit4]   2> 1190565 T2963 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1190565 T2973 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1190565 T2986 oascc.ZkStateReader$3.process Updating live nodes... (2)
   [junit4]   2> 1190584 T2987 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 1190585 T2987 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1190586 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1190586 T2987 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1190587 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1190587 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1190588 T2964 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1190589 T2964 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:16261",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:16261_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1190589 T2964 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 1190589 T2964 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 1190593 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1190594 T2986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1190594 T2963 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1190594 T2973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1191587 T2987 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1191588 T2987 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 1191589 T2987 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1191589 T2987 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1191590 T2987 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1191590 T2987 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-003/collection1/'
   [junit4]   2> 1191591 T2987 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 1191592 T2987 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 1191593 T2987 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 1191660 T2987 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
   [junit4]   2> 1191708 T2987 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1191709 T2987 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1191722 T2987 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1192175 T2987 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1192175 T2987 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1192176 T2987 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1192189 T2987 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1192192 T2987 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1192205 T2987 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1192210 T2987 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1192214 T2987 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1192215 T2987 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1192215 T2987 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1192216 T2987 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1192217 T2987 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1192217 T2987 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1192217 T2987 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1192217 T2987 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1192218 T2987 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-003/collection1/, dataDir=./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/jetty1/
   [junit4]   2> 1192218 T2987 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@50fe766d
   [junit4]   2> 1192219 T2987 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/jetty1
   [junit4]   2> 1192219 T2987 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/jetty1/index/
   [junit4]   2> 1192219 T2987 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 1192220 T2987 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/jetty1/index
   [junit4]   2> 1192220 T2987 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=91.1650390625, floorSegmentMB=0.2197265625, forceMergeDeletesPctAllowed=7.796980668834749, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1192221 T2987 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3ed05ef6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@fbe3079),segFN=segments_1,generation=1}
   [junit4]   2> 1192221 T2987 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1192227 T2987 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1192227 T2987 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1192228 T2987 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1192228 T2987 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1192228 T2987 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1192229 T2987 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1192229 T2987 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1192229 T2987 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1192230 T2987 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1192230 T2987 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1192231 T2987 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1192231 T2987 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1192232 T2987 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1192232 T2987 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1192233 T2987 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1192233 T2987 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1192252 T2987 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1192257 T2987 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1192257 T2987 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1192258 T2987 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=50, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7941882430337752]
   [junit4]   2> 1192258 T2987 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@3ed05ef6 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@fbe3079),segFN=segments_1,generation=1}
   [junit4]   2> 1192259 T2987 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1192259 T2987 oass.SolrIndexSearcher.<init> Opening Searcher@d2af779[collection1] main
   [junit4]   2> 1192259 T2987 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1192260 T2987 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1192260 T2987 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1192261 T2987 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1192261 T2987 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1192261 T2987 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1192262 T2987 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1192262 T2987 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1192263 T2987 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 1192263 T2987 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1192267 T2988 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@d2af779[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1192270 T2987 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1192270 T2991 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:16261 collection:collection1 shard:shard2
   [junit4]   2> 1192271 T2941 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
   [junit4]   2> 1192271 T2941 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1192271 T2991 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 1192290 T2991 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 1192300 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1192300 T2991 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1192301 T2991 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C912 name=collection1 org.apache.solr.core.SolrCore@59e8043f url=http://127.0.0.1:16261/collection1 node=127.0.0.1:16261_ C912_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:16261, core=collection1, node_name=127.0.0.1:16261_}
   [junit4]   2> 1192301 T2991 C912 P16261 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:16261/collection1/
   [junit4]   2> 1192301 T2991 C912 P16261 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1192301 T2964 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1192301 T2991 C912 P16261 oasc.SyncStrategy.syncToMe http://127.0.0.1:16261/collection1/ has no replicas
   [junit4]   2> 1192302 T2991 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:16261/collection1/ shard2
   [junit4]   2> 1192302 T2991 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 1192311 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1192313 T2963 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1192313 T2973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1192313 T2986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1192325 T2964 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1192329 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1192432 T2963 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1192432 T2973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1192432 T2986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1192477 T2991 oasc.ZkController.register We are http://127.0.0.1:16261/collection1/ and leader is http://127.0.0.1:16261/collection1/
   [junit4]   2> 1192478 T2991 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:16261
   [junit4]   2> 1192478 T2991 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1192478 T2991 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 1192478 T2991 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1192480 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1192480 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1192480 T2991 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1192480 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1192482 T2964 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1192482 T2964 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:16261",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:16261_",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1"}
   [junit4]   2> 1192491 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1192594 T2963 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1192594 T2986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1192594 T2973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 1193318 T2941 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 1193319 T2941 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1193321 T2941 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:18801
   [junit4]   2> 1193322 T2941 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1193322 T2941 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1193322 T2941 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-004
   [junit4]   2> 1193323 T2941 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-004/'
   [junit4]   2> 1193356 T2941 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-004/solr.xml
   [junit4]   2> 1193409 T2941 oasc.CoreContainer.<init> New CoreContainer 1621405229
   [junit4]   2> 1193409 T2941 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-004/]
   [junit4]   2> 1193410 T2941 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1193410 T2941 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1193411 T2941 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1193411 T2941 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1193411 T2941 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1193411 T2941 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1193412 T2941 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1193412 T2941 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1193412 T2941 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1193413 T2941 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1193413 T2941 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1193413 T2941 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1193414 T2941 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:15777/solr
   [junit4]   2> 1193414 T2941 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1193415 T2941 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1193417 T3003 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5ba73dec name:ZooKeeperConnection Watcher:127.0.0.1:15777 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1193418 T2941 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1193433 T2941 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1193435 T3005 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@26f61081 name:ZooKeeperConnection Watcher:127.0.0.1:15777/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1193435 T2941 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1193443 T2941 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1194447 T2941 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44776_
   [junit4]   2> 1194477 T2941 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44776_
   [junit4]   2> 1194481 T2973 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1194482 T2963 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1194482 T3005 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1194482 T2986 oascc.ZkStateReader$3.process Updating live nodes... (3)
   [junit4]   2> 1194498 T3006 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 1194498 T3006 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1194500 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1194500 T3006 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1194500 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1194501 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1194502 T2964 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1194503 T2964 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:44776",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:44776_",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":null}
   [junit4]   2> 1194503 T2964 oasc.Overseer$ClusterStateUpdater.updateState Collection already exists with numShards=2
   [junit4]   2> 1194503 T2964 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 1194514 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1194515 T2963 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1194515 T2986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1194515 T2973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1194515 T3005 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1195501 T3006 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1195502 T3006 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 1195502 T3006 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1195503 T3006 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1195504 T3006 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1195504 T3006 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: './solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-004/collection1/'
   [junit4]   2> 1195505 T3006 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-004/collection1/lib/README' to classloader
   [junit4]   2> 1195506 T3006 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-004/collection1/lib/classes/' to classloader
   [junit4]   2> 1195506 T3006 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-004/collection1/lib/.svn/' to classloader
   [junit4]   2> 1195574 T3006 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_4_10
   [junit4]   2> 1195623 T3006 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1195624 T3006 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1195636 T3006 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 1196072 T3006 oass.ByteField.init WARN ByteField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1196073 T3006 oass.ShortField.init WARN ShortField is deprecated and will be removed in 5.0. You should use TrieIntField instead.
   [junit4]   2> 1196074 T3006 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1196088 T3006 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1196091 T3006 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1196104 T3006 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1196109 T3006 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1196113 T3006 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1196114 T3006 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1196114 T3006 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1196114 T3006 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1196115 T3006 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1196115 T3006 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1196116 T3006 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 1196116 T3006 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1196116 T3006 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-004/collection1/, dataDir=./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/jetty2/
   [junit4]   2> 1196117 T3006 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@50fe766d
   [junit4]   2> 1196117 T3006 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/jetty2
   [junit4]   2> 1196118 T3006 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/jetty2/index/
   [junit4]   2> 1196118 T3006 oasc.SolrCore.initIndex WARN [collection1] Solr index directory './solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 1196118 T3006 oasc.CachingDirectoryFactory.get return new directory for ./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001/tempDir-001/jetty2/index
   [junit4]   2> 1196119 T3006 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=91.1650390625, floorSegmentMB=0.2197265625, forceMergeDeletesPctAllowed=7.796980668834749, segmentsPerTier=19.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 1196120 T3006 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@42a1c464 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@40e79950),segFN=segments_1,generation=1}
   [junit4]   2> 1196120 T3006 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1196126 T3006 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1196126 T3006 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1196127 T3006 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1196127 T3006 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1196127 T3006 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1196127 T3006 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1196128 T3006 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1196128 T3006 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1196129 T3006 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1196129 T3006 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 1196130 T3006 oasc.RequestHandlers.initHandlersFromConfig created /get: solr.RealTimeGetHandler
   [junit4]   2> 1196130 T3006 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.ReplicationHandler
   [junit4]   2> 1196131 T3006 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 1196131 T3006 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1196132 T3006 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1196132 T3006 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 1196151 T3006 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1196156 T3006 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1196156 T3006 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1196157 T3006 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=50, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7941882430337752]
   [junit4]   2> 1196158 T3006 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@42a1c464 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@40e79950),segFN=segments_1,generation=1}
   [junit4]   2> 1196158 T3006 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1196158 T3006 oass.SolrIndexSearcher.<init> Opening Searcher@88e6744[collection1] main
   [junit4]   2> 1196159 T3006 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 1196160 T3006 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1196160 T3006 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1196160 T3006 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1196161 T3006 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 1196161 T3006 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1196162 T3006 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1196162 T3006 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1196162 T3006 oasr.ManagedResource.notifyObserversDuringInit WARN No registered observers for /rest/managed
   [junit4]   2> 1196162 T3006 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 1196167 T3007 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@88e6744[collection1] main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1196170 T3006 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 1196171 T3010 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44776 collection:collection1 shard:shard1
   [junit4]   2> 1196171 T2941 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1
   [junit4]   2> 1196171 T2941 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1196172 T3010 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 1196173 T2941 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1196174 T2941 oasc.AbstractFullDistribZkTestBase.waitForThingsToLevelOut Wait for recoveries to finish - wait 30000 for each attempt
   [junit4]   2> 1196174 T2941 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30000
   [junit4]   2> 1196200 T2941 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1196213 T3010 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 1196217 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1196217 T3010 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 1196217 T3010 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C913 name=collection1 org.apache.solr.core.SolrCore@3da63b68 url=http://127.0.0.1:44776/collection1 node=127.0.0.1:44776_ C913_STATE=coll:collection1 core:collection1 props:{state=down, base_url=http://127.0.0.1:44776, core=collection1, node_name=127.0.0.1:44776_}
   [junit4]   2> 1196217 T3010 C913 P44776 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44776/collection1/
   [junit4]   2> 1196218 T3010 C913 P44776 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 1196218 T2964 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1196218 T3010 C913 P44776 oasc.SyncStrategy.syncToMe http://127.0.0.1:44776/collection1/ has no replicas
   [junit4]   2> 1196218 T3010 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44776/collection1/ shard1
   [junit4]   2> 1196219 T3010 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 1196227 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1196228 T2963 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1196229 T3005 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1196228 T2973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1196228 T2986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1196239 T2964 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1196242 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1196344 T2986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1196344 T2963 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1196344 T3005 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1196344 T2973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1196390 T3010 oasc.ZkController.register We are http://127.0.0.1:44776/collection1/ and leader is http://127.0.0.1:44776/collection1/
   [junit4]   2> 1196391 T3010 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44776
   [junit4]   2> 1196391 T3010 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 1196391 T3010 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 1196391 T3010 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 1196393 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1196393 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1196393 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1196393 T3010 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1196395 T2964 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1196395 T2964 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:44776",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"127.0.0.1:44776_",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node2"}
   [junit4]   2> 1196404 T2963 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1196507 T2973 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1196507 T2963 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1196507 T3005 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1196507 T2986 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 1197203 T2941 oascc.ZkStateReader.updateClusterState Updating cloud state from ZooKeeper... 
   [junit4]   2> 1197204 T2941 oasc.AbstractDistribZkTestBase.waitForRecoveriesToFinish Recoveries finished - collection: collection1
   [junit4]   2> 1197206 T2971 oasc.SocketProxy$Acceptor.run accepted Socket[addr=/140.211.11.196,port=44097,localport=12802], receiveBufferSize:6565888
   [junit4]   2> 1197210 T2971 oasc.SocketProxy$Bridge.<init> proxy connection Socket[addr=/127.0.0.1,port=36008,localport=45082], receiveBufferSize=6565888
   [junit4]   2> ASYNC  NEW_CORE C914 name=collection1 org.apache.solr.core.SolrCore@3994bee8 url=http://127.0.0.1:12802/collection1 node=127.0.0.1:12802_ C914_STATE=coll:control_collection core:collection1 props:{state=active, base_url=http://127.0.0.1:12802, c

[...truncated too long message...]

_1x2_shard1_replica1",
   [junit4]    >           "node_name":"127.0.0.1:44776_"},
   [junit4]    >         "core_node2":{
   [junit4]    >           "state":"active",
   [junit4]    >           "base_url":"http://127.0.0.1:12802",
   [junit4]    >           "core":"c8n_1x2_shard1_replica2",
   [junit4]    >           "node_name":"127.0.0.1:12802_",
   [junit4]    >           "leader":"true"}}}}}}
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2F6FD45E5446A9CF:AE895A462319C9F3]:0)
   [junit4]    > 	at org.apache.solr.cloud.HttpPartitionTest.ensureAllReplicasAreActive(HttpPartitionTest.java:505)
   [junit4]    > 	at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:215)
   [junit4]    > 	at org.apache.solr.cloud.HttpPartitionTest.doTest(HttpPartitionTest.java:142)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:865)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 1398010 T2941 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J1/./solr.cloud.HttpPartitionTest-2F6FD45E5446A9CF-001
   [junit4]   2> 212756 T2940 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 6 leaked thread(s).
   [junit4]   2> 1398225 T3092 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
   [junit4]   2> 
   [junit4]   2> 1398226 T2963 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 1398298 T3095 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
   [junit4]   2> 
   [junit4]   2> 1398299 T2986 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 1399290 T3005 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 1399290 T3099 oasc.Overseer$ClusterStateUpdater.checkIfIamStillLeader ERROR could not read the data org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /overseer_elect/leader
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
   [junit4]   2> 	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
   [junit4]   2> 	at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1155)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:277)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:73)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:274)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:316)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.access$300(Overseer.java:85)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater$1.run(Overseer.java:288)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=Lucene49: {intDefault=PostingsFormat(name=FSTOrdPulsing41), a_t=PostingsFormat(name=FSTOrdPulsing41), _version_=PostingsFormat(name=FSTOrdPulsing41), text=PostingsFormat(name=Lucene41WithOrds), range_facet_sl=PostingsFormat(name=Lucene41WithOrds), multiDefault=PostingsFormat(name=Asserting), range_facet_si=PostingsFormat(name=Asserting), id=PostingsFormat(name=Direct), timestamp=PostingsFormat(name=FSTOrdPulsing41), range_facet_l=PostingsFormat(name=FSTOrdPulsing41)}, docValues:{}, sim=DefaultSimilarity, locale=hu, timezone=Asia/Vladivostok
   [junit4]   2> NOTE: FreeBSD 9.1-RELEASE-p3 amd64/Oracle Corporation 1.7.0_60 (64-bit)/cpus=16,threads=1,free=357669336,total=535822336
   [junit4]   2> NOTE: All tests run in this JVM: [TestPostingsSolrHighlighter, TestSolrQueryParserResource, ShardSplitTest, TestTrie, TestLuceneMatchVersion, DocumentBuilderTest, DirectUpdateHandlerTest, TestDefaultSearchFieldResource, TestCollapseQParserPlugin, BadCopyFieldTest, TestSchemaNameResource, TestCollationField, TestDistributedMissingSort, TestBlendedInfixSuggestions, TestReload, NotRequiredUniqueKeyTest, LoggingHandlerTest, SchemaVersionSpecificBehaviorTest, TestSchemaSimilarityResource, DistributedQueryElevationComponentTest, TestFieldTypeResource, BasicDistributedZk2Test, AlternateDirectoryTest, TestFaceting, DistributedExpandComponentTest, SpellingQueryConverterTest, CoreContainerCoreInitFailuresTest, TestSweetSpotSimilarityFactory, TestFiltering, TestQuerySenderListener, TestCodecSupport, CoreMergeIndexesAdminHandlerTest, RequestHandlersTest, TestBinaryResponseWriter, BasicFunctionalityTest, TestSolrIndexConfig, AutoCommitTest, BasicDistributedZkTest, TestCSVResponseWriter, OpenCloseCoreStressTest, BinaryUpdateRequestHandlerTest, ZkSolrClientTest, TestAnalyzeInfixSuggestions, DistribDocExpirationUpdateProcessorTest, TermsComponentTest, HdfsRecoveryZkTest, SpellCheckComponentTest, SolrIndexSplitterTest, FileBasedSpellCheckerTest, TestRecovery, TestLRUCache, TestQuerySenderNoQuery, SolrRequestParserTest, CursorPagingTest, TestDynamicFieldResource, TestHighFrequencyDictionaryFactory, AssignTest, QueryResultKeyTest, SearchHandlerTest, TestInfoStreamLogging, TestInitQParser, HdfsBasicDistributedZkTest, TestFieldTypeCollectionResource, FastVectorHighlighterTest, TestFieldCollectionResource, DebugComponentTest, TestCSVLoader, StatelessScriptUpdateProcessorFactoryTest, SuggesterFSTTest, SolrPluginUtilsTest, TestNonNRTOpen, EnumFieldTest, OverseerStatusTest, PrimUtilsTest, TestRemoteStreaming, HardAutoCommitTest, TestXIncludeConfig, TestJmxIntegration, ShardRoutingTest, TestNumberUtils, TestDocBasedVersionConstraints, TestDefaultSimilarityFactory, BadComponentTest, TestRandomDVFaceting, DistributedTermsComponentTest, TestCloudInspectUtil, URLClassifyProcessorTest, TestRTGBase, TestUpdate, HttpPartitionTest]
   [junit4] Completed on J1 in 214.32s, 1 test, 1 failure <<< FAILURES!

[...truncated 981 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:447: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:45: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/extra-targets.xml:37: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:496: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1299: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:923: There were test failures: 404 suites, 1654 tests, 1 failure, 51 ignored (17 assumptions)

Total time: 105 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-4.x-Java7 #2008
Archived 3 artifacts
Archive block size is 32768
Received 0 blocks and 192399174 bytes
Compression is 0.0%
Took 35 sec
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure



[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 2018 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/2018/

All tests passed

Build Log:
[...truncated 29242 lines...]
check-licenses:
     [echo] License check under: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr
 [licenses] MISSING sha1 checksum file for: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/example/lib/ext/log4j-1.2.16.jar
 [licenses] EXPECTED sha1 checksum file : /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/licenses/log4j-1.2.16.jar.sha1

[...truncated 1 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:467: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:70: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:254: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/tools/custom-tasks.xml:62: License check failed. Check the logs.

Total time: 135 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Sending artifact delta relative to Lucene-Solr-Tests-4.x-Java7 #2008
Archived 1 artifacts
Archive block size is 32768
Received 0 blocks and 464 bytes
Compression is 0.0%
Took 14 ms
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure