You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/11/30 11:07:37 UTC

[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_20) - Build # 4461 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4461/
Java: 32bit/jdk1.8.0_20 -server -XX:+UseParallelGC (asserts: false)

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DeleteInactiveReplicaTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.DeleteInactiveReplicaTest:     1) Thread[id=1884, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteInactiveReplicaTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.DeleteInactiveReplicaTest: 
   1) Thread[id=1884, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteInactiveReplicaTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DeleteReplicaTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest:     1) Thread[id=13675, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest: 
   1) Thread[id=13675, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)


FAILED:  org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)




Build Log:
[...truncated 8643 lines...]
   [junit4] Suite: org.apache.solr.cloud.DeleteInactiveReplicaTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\init-core-data-001
   [junit4]   2> 662906 T1881 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 662906 T1881 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /p_gg/dv
   [junit4]   2> 662908 T1881 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 662908 T1881 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 662909 T1882 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 663010 T1881 oasc.ZkTestServer.run start zk server on port:50887
   [junit4]   2> 663010 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 663012 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 663016 T1889 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c4c909 name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663016 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 663016 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 663016 T1881 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 663023 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 663025 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 663028 T1892 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b55e8b name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663029 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 663029 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 663029 T1881 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 663032 T1881 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 663035 T1881 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 663037 T1881 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 663039 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 663039 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 663043 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 663043 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 663046 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 663046 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 663049 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 663049 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 663052 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 663052 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 663055 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 663055 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 663058 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 663058 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 663061 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 663061 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 663064 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 663064 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 663067 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 663067 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 663070 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 663070 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 663074 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 663076 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 663081 T1895 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1997f53 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663081 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 663081 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 664215 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 664223 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 664224 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50897
   [junit4]   2> 664228 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 664228 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 664229 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002
   [junit4]   2> 664229 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002\'
   [junit4]   2> 664255 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002\solr.xml
   [junit4]   2> 664266 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002
   [junit4]   2> 664267 T1881 oasc.CoreContainer.<init> New CoreContainer 14486680
   [junit4]   2> 664267 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002\]
   [junit4]   2> 664267 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 664268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 664268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 664269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 664269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 664269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 664269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 664270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 664270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 664270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 664274 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 664277 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 664277 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 664277 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 664278 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 664278 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 664278 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 664290 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 664294 T1907 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a185aa name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 664295 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 664295 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 664301 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 664304 T1910 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1855eb0 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 664304 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 664307 T1881 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 664312 T1881 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 664317 T1881 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 664322 T1881 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 664326 T1881 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 664333 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 664336 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50897_p_gg%2Fdv
   [junit4]   2> 664337 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50897_p_gg%2Fdv
   [junit4]   2> 664341 T1881 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 664345 T1881 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 664348 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 664351 T1881 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:50897_p_gg%2Fdv
   [junit4]   2> 664352 T1881 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 664355 T1881 oasc.Overseer.start Overseer (id=92886067165528068-127.0.0.1:50897_p_gg%2Fdv-n_0000000000) starting
   [junit4]   2> 664359 T1881 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 664377 T1881 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 664380 T1881 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 664380 T1912 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 664384 T1881 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 664387 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 664393 T1911 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 664393 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002
   [junit4]   2> 664701 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002\conf\
   [junit4]   2> 664756 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 664758 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 664758 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 664778 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 664781 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 664783 T1917 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11cda57 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 664783 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 664784 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 664787 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 664793 T1881 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 666145 T1881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 666146 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 666155 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 666156 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50909
   [junit4]   2> 666159 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 666160 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 666160 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003
   [junit4]   2> 666160 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\'
   [junit4]   2> 666191 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\solr.xml
   [junit4]   2> 666201 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003
   [junit4]   2> 666202 T1881 oasc.CoreContainer.<init> New CoreContainer 11913574
   [junit4]   2> 666202 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\]
   [junit4]   2> 666203 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 666203 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 666203 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 666204 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 666204 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 666204 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 666205 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 666205 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 666205 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 666206 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 666209 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 666211 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 666211 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 666211 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 666212 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 666212 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 666212 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 666213 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 666229 T1929 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@115393e name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 666230 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 666230 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 666232 T1883 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x149ff62cdfc0006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 666234 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 666240 T1932 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fa8465 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 666240 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 666250 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 667259 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50909_p_gg%2Fdv
   [junit4]   2> 667261 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50909_p_gg%2Fdv
   [junit4]   2> 667272 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 667277 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003
   [junit4]   2> 667960 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\conf\
   [junit4]   2> 668075 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 668077 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 668077 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 669281 T1881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 669281 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 669286 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 669288 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50918
   [junit4]   2> 669292 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 669293 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 669293 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004
   [junit4]   2> 669293 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\'
   [junit4]   2> 669324 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\solr.xml
   [junit4]   2> 669335 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004
   [junit4]   2> 669336 T1881 oasc.CoreContainer.<init> New CoreContainer 19289259
   [junit4]   2> 669336 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\]
   [junit4]   2> 669337 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 669337 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 669337 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 669338 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 669338 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 669338 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 669338 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 669339 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 669339 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 669339 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 669343 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 669346 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 669347 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 669347 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 669347 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 669348 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 669348 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 669351 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 669353 T1945 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ee49f name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 669354 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 669354 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 669358 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 669359 T1883 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x149ff62cdfc0008, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 669362 T1948 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67030b name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 669362 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 669374 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 670381 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50918_p_gg%2Fdv
   [junit4]   2> 670383 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50918_p_gg%2Fdv
   [junit4]   2> 670393 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 670398 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004
   [junit4]   2> 670949 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\conf\
   [junit4]   2> 671009 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 671011 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 671011 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 672200 T1881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 672201 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 672207 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 672209 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50927
   [junit4]   2> 672214 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 672214 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 672215 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005
   [junit4]   2> 672215 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\'
   [junit4]   2> 672252 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\solr.xml
   [junit4]   2> 672266 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005
   [junit4]   2> 672267 T1881 oasc.CoreContainer.<init> New CoreContainer 7043761
   [junit4]   2> 672267 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\]
   [junit4]   2> 672268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 672268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 672268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 672269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 672269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 672269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 672270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 672270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 672270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 672271 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 672280 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 672282 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 672283 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 672283 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 672284 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 672284 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 672284 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 672286 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 672290 T1961 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c4587 name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 672291 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 672291 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 672296 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 672300 T1964 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dd9180 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 672301 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 672314 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 673323 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50927_p_gg%2Fdv
   [junit4]   2> 673324 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50927_p_gg%2Fdv
   [junit4]   2> 673339 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 673344 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005
   [junit4]   2> 673974 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\conf\
   [junit4]   2> 674141 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 674142 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 674143 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 675358 T1881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 675358 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 675366 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 675368 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50936
   [junit4]   2> 675373 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 675373 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 675373 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006
   [junit4]   2> 675373 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\'
   [junit4]   2> 675398 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\solr.xml
   [junit4]   2> 675412 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006
   [junit4]   2> 675413 T1881 oasc.CoreContainer.<init> New CoreContainer 31822724
   [junit4]   2> 675413 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\]
   [junit4]   2> 675413 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 675415 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 675415 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 675417 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 675419 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 675419 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 675419 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 675419 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 675420 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 675420 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 675421 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 675425 T1977 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2092d5 name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 675426 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 675426 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 675431 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 675433 T1980 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@110b7ca name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 675434 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 675444 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 676449 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50936_p_gg%2Fdv
   [junit4]   2> 676452 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50936_p_gg%2Fdv
   [junit4]   2> 676464 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 676472 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006
   [junit4]   2> 677561 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\conf\
   [junit4]   2> 677778 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 677780 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 677781 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 677787 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 677788 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 677791 T1984 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@87324c name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 677792 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 677792 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 677794 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 677852 T1902 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=clusterprop&wt=javabin&version=2} status=0 QTime=6 
   [junit4]   2> 677853 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 677859 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 0, message = {
   [junit4]   2> 	  "operation":"clusterprop",
   [junit4]   2> 	  "name":"legacyCloud",
   [junit4]   2> 	  "val":"false"}
   [junit4]   2> 677864 T1911 oasco.CollectionMutator.checkKeyExistence ERROR Skipping invalid Overseer message because it has no collection specified: {
   [junit4]   2> 	  "operation":"clusterprop",
   [junit4]   2> 	  "name":"legacyCloud",
   [junit4]   2> 	  "val":"false"}
   [junit4]   2> 677867 T1911 oasc.Overseer$ClusterStateUpdater.run ERROR Overseer could not process the current clusterstate state update message, skipping the message. org.apache.solr.common.SolrException: Overseer SETREPLICAPROPERTY requires collection and shard and replica and property and property.value no action taken.
   [junit4]   2> 	at org.apache.solr.cloud.overseer.ReplicaMutator.addReplicaProperty(ReplicaMutator.java:96)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:444)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:323)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 677870 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 677872 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 677876 T1987 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@129fc43 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 677876 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 677877 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 677879 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 677894 T1904 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&maxShardsPerNode=2&name=delDeadColl&action=CREATE&numShards=2&wt=javabin&version=2
   [junit4]   2> 677896 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/collection-queue-work state SyncConnected
   [junit4]   2> 677897 T1912 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delDeadColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 677898 T1988 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delDeadColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 677902 T1988 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
   [junit4]   2> 677902 T1988 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/delDeadColl 
   [junit4]   2> 677903 T1988 oascc.SolrZkClient.makePath makePath: /collections/delDeadColl
   [junit4]   2> 677908 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 677911 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delDeadColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 677911 T1911 oasco.ClusterStateMutator.createCollection building a new cName: delDeadColl
   [junit4]   2> 677917 T1948 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677918 T1910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677918 T1964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677919 T1980 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677920 T1932 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677920 T1984 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677926 T1917 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677926 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678009 T1988 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection delDeadColl, shardNames [shard1, shard2] , replicationFactor : 2
   [junit4]   2> 678009 T1988 oasc.OverseerCollectionProcessor.createCollection Creating shard delDeadColl_shard1_replica1 as part of slice shard1 of collection delDeadColl on 127.0.0.1:50909_p_gg%2Fdv
   [junit4]   2> 678013 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678014 T1988 oasc.OverseerCollectionProcessor.createCollection Creating shard delDeadColl_shard1_replica2 as part of slice shard1 of collection delDeadColl on 127.0.0.1:50936_p_gg%2Fdv
   [junit4]   2> 678016 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50909/p_gg/dv"}
   [junit4]   2> 678016 T1911 oasco.SliceMutator.addReplica createReplica() {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50909/p_gg/dv"} 
   [junit4]   2> 678021 T1988 oasc.OverseerCollectionProcessor.createCollection Creating shard delDeadColl_shard2_replica1 as part of slice shard2 of collection delDeadColl on 127.0.0.1:50927_p_gg%2Fdv
   [junit4]   2> 678024 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50936/p_gg/dv"}
   [junit4]   2> 678025 T1911 oasco.SliceMutator.addReplica createReplica() {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50936/p_gg/dv"} 
   [junit4]   2> 678027 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678027 T1988 oasc.OverseerCollectionProcessor.createCollection Creating shard delDeadColl_shard2_replica2 as part of slice shard2 of collection delDeadColl on 127.0.0.1:50918_p_gg%2Fdv
   [junit4]   2> 678031 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50927/p_gg/dv"}
   [junit4]   2> 678032 T1911 oasco.SliceMutator.addReplica createReplica() {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50927/p_gg/dv"} 
   [junit4]   2> 678033 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678035 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50918/p_gg/dv"}
   [junit4]   2> 678036 T1911 oasco.SliceMutator.addReplica createReplica() {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50918/p_gg/dv"} 
   [junit4]   2> 678039 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678041 T1948 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678041 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678041 T1917 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678041 T1964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678042 T1980 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678042 T1984 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678042 T1910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678042 T1932 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678197 T1974 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&name=delDeadColl_shard1_replica2&action=CREATE&numShards=2&collection=delDeadColl&shard=shard1&wt=javabin&version=2
   [junit4]   2> 678226 T1957 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&name=delDeadColl_shard2_replica1&action=CREATE&numShards=2&collection=delDeadColl&shard=shard2&wt=javabin&version=2
   [junit4]   2> 678239 T1940 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&name=delDeadColl_shard2_replica2&action=CREATE&numShards=2&collection=delDeadColl&shard=shard2&wt=javabin&version=2
   [junit4]   2> 678266 T1923 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&coreNodeName=core_node1&collection.configName=conf1&name=delDeadColl_shard1_replica1&action=CREATE&numShards=2&collection=delDeadColl&shard=shard1&wt=javabin&version=2
   [junit4]   2> 678342 T1974 oasc.ZkController.publish publishing core=delDeadColl_shard1_replica2 state=down collection=delDeadColl
   [junit4]   2> 678343 T1974 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delDeadColl_shard1_replica2
   [junit4]   2> 678344 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678345 T1957 oasc.ZkController.publish publishing core=delDeadColl_shard2_replica1 state=down collection=delDeadColl
   [junit4]   2> 678346 T1957 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delDeadColl_shard2_replica1
   [junit4]   2> 678347 T1940 oasc.ZkController.publish publishing core=delDeadColl_shard2_replica2 state=down collection=delDeadColl
   [junit4]   2> 678347 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica2",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50936/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50936_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678349 T1940 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delDeadColl_shard2_replica2
   [junit4]   2> 678349 T1911 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica2",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50936/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50936_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678355 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 2, message = {
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica1",
   [junit4]   2> 	  "core_node_name":"core_node3",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50927/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50927_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678358 T1911 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica1",
   [junit4]   2> 	  "core_node_name":"core_node3",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50927/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50927_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678362 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678364 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 2, message = {
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica2",
   [junit4]   2> 	  "core_node_name":"core_node4",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50918/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50918_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678365 T1911 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica2",
   [junit4]   2> 	  "core_node_name":"core_node4",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50918/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50918_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678370 T1923 oasc.ZkController.publish publishing core=delDeadColl_shard1_replica1 state=down collection=delDeadColl
   [junit4]   2> 678371 T1923 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delDeadColl_shard1_replica1
   [junit4]   2> 678372 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678373 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50909/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50909_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678375 T1911 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50909/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50909_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678481 T1948 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1984 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678483 T1917 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1932 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1980 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 679344 T1974 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delDeadColl
   [junit4]   2> 679345 T1974 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 679345 T1974 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679346 T1957 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delDeadColl
   [junit4]   2> 679347 T1957 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 679348 T1957 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679348 T1974 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679349 T1974 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\'
   [junit4]   2> 679349 T1940 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delDeadColl
   [junit4]   2> 679352 T1940 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 679352 T1940 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679353 T1957 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679357 T1957 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\delDeadColl_shard2_replica1\'
   [junit4]   2> 679362 T1940 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679362 T1940 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\'
   [junit4]   2> 679373 T1923 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delDeadColl
   [junit4]   2> 679379 T1923 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 679379 T1923 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679383 T1923 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679383 T1923 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\delDeadColl_shard1_replica1\'
   [junit4]   2> 679391 T1974 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 679410 T1940 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 679414 T1974 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 679419 T1923 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 679419 T1940 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 679425 T1957 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 679436 T1974 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 679439 T1974 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 679447 T1940 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 679449 T1940 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 679450 T1923 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 679451 T1957 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 679452 T1974 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 679455 T1974 oass.IndexSchema.readSchema [delDeadColl_shard1_replica2] Schema name=test
   [junit4]   2> 679470 T1940 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 679474 T1940 oass.IndexSchema.readSchema [delDeadColl_shard2_replica2] Schema name=test
   [junit4]   2> 679493 T1923 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 679499 T1923 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 679503 T1957 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 679507 T1957 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 679526 T1974 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 679538 T1974 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679540 T1974 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679543 T1923 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 679547 T1923 oass.IndexSchema.readSchema [delDeadColl_shard1_replica1] Schema name=test
   [junit4]   2> 679573 T1940 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 679576 T1940 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679577 T1940 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679580 T1957 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 679592 T1957 oass.IndexSchema.readSchema [delDeadColl_shard2_replica1] Schema name=test
   [junit4]   2> 679661 T1923 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 679663 T1957 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 679663 T1923 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679664 T1923 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679666 T1957 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679667 T1957 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679695 T1974 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679700 T1974 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679706 T1974 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679709 T1940 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679710 T1974 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679710 T1974 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679710 T1974 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679712 T1923 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679715 T1940 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679716 T1957 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679717 T1974 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679717 T1974 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679717 T1974 oasc.CoreContainer.create Creating SolrCore 'delDeadColl_shard1_replica2' using configuration from collection delDeadColl
   [junit4]   2> 679717 T1974 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 679718 T1974 oasc.SolrCore.<init> [delDeadColl_shard1_replica2] Opening new SolrCore at ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\
   [junit4]   2> 679718 T1974 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c96c8
   [junit4]   2> 679719 T1974 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\
   [junit4]   2> 679719 T1974 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\index/
   [junit4]   2> 679719 T1974 oasc.SolrCore.initIndex WARN [delDeadColl_shard1_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\index' doesn't exist. Creating new index...
   [junit4]   2> 679720 T1974 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\index
   [junit4]   2> 679720 T1974 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8867238024671883]
   [junit4]   2> 679722 T1974 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@19be8be lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b83ff1),segFN=segments_1,generation=1}
   [junit4]   2> 679722 T1974 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 679723 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 679724 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 679724 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 679724 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 679725 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 679725 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 679725 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 679726 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 679726 T1974 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 679726 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 679727 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 679727 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 679727 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 679728 T1974 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 679728 T1974 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 679729 T1974 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 679729 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 679730 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 679730 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 679731 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 679731 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 679731 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 679732 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 679732 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 679732 T1974 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 679733 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 679733 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 679738 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679739 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679740 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679740 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679744 T1974 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 679744 T1974 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 679745 T1974 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=4, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.45034392590936567]
   [junit4]   2> 679745 T1974 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@19be8be lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b83ff1),segFN=segments_1,generation=1}
   [junit4]   2> 679745 T1974 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 679746 T1974 oass.SolrIndexSearcher.<init> Opening Searcher@b40dcf[delDeadColl_shard1_replica2] main
   [junit4]   2> 679746 T1974 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679749 T1923 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679752 T1940 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679754 T1957 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679756 T1974 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679756 T1974 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 679758 T1923 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679759 T1974 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 679759 T1974 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 679759 T1974 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 679760 T1940 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679760 T1940 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679760 T1940 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679762 T1957 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679763 T1974 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 679763 T1974 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 679765 T1923 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679765 T1923 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679765 T1923 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679766 T1974 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
   [junit4]   2> 679767 T1940 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679767 T1940 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679767 T1940 oasc.CoreContainer.create Creating SolrCore 'delDeadColl_shard2_replica2' using configuration from collection delDeadColl
   [junit4]   2> 679767 T1940 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 679768 T1940 oasc.SolrCore.<init> [delDeadColl_shard2_replica2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\data\
   [junit4]   2> 679768 T1940 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c96c8
   [junit4]   2> 679768 T1940 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\data\
   [junit4]   2> 679768 T1940 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\data\index/
   [junit4]   2> 679768 T1940 oasc.SolrCore.initIndex WARN [delDeadColl_shard2_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplica

[...truncated too long message...]

Schema, CurrencyFieldXmlFileTest, TestQueryUtils, OverseerStatusTest, SpellPossibilityIteratorTest, ResponseLogComponentTest, TestOmitPositions, DistributedFacetPivotSmallAdvancedTest, MergeStrategyTest, TestLMJelinekMercerSimilarityFactory, TestInitQParser, DateRangeFieldTest, TestSweetSpotSimilarityFactory, TestFileDictionaryLookup, RemoteQueryErrorTest, CustomCollectionTest, TestLRUCache, SpatialFilterTest, DistributedSpellCheckComponentTest, ChangedSchemaMergeTest, BJQParserTest, ChaosMonkeySafeLeaderTest, DateMathParserTest, TestRealTimeGet, AddSchemaFieldsUpdateProcessorFactoryTest, TestDistributedSearch, TestDistributedMissingSort, HdfsBasicDistributedZkTest, SuggesterTSTTest, SolrIndexConfigTest, TestMissingGroups, TestUpdate, TestBM25SimilarityFactory, TestSolrQueryParserDefaultOperatorResource, TestXIncludeConfig, UniqFieldsUpdateProcessorFactoryTest, TestAddFieldRealTimeGet, TestCodecSupport, TestSolrXmlPersistor, FileBasedSpellCheckerTest, HdfsUnloadDistributedZkTest, TestFastOutputStream, TestCustomSort, TestSolr4Spatial2, TestAnalyzeInfixSuggestions, TestExpandComponent, TestSchemaManager, TestQuerySenderListener, AnalysisErrorHandlingTest, CloudExitableDirectoryReaderTest, TermVectorComponentTest, TestStressVersions, SliceStateTest, SolrCoreTest, MBeansHandlerTest, TestPivotHelperCode, LeaderFailoverAfterPartitionTest, HighlighterMaxOffsetTest, TestClassNameShortening, ResourceLoaderTest, TestSolrJ, TestSolrConfigHandler, PeerSyncTest, TestLuceneMatchVersion, TestWriterPerf, OpenExchangeRatesOrgProviderTest, TestIndexingPerformance, TestManagedStopFilterFactory, TestStressRecovery, TestDynamicFieldCollectionResource, PingRequestHandlerTest, TestStandardQParsers, AnalysisAfterCoreReloadTest, TestModifyConfFiles, TestSolrCoreProperties, TestQueryTypes, TriLevelCompositeIdRoutingTest, ConvertedLegacyTest, DistributedFacetPivotLargeTest, ScriptEngineTest, CoreAdminHandlerTest, DistributedIntervalFacetingTest, TestSystemIdResolver, TestTrie, PrimUtilsTest, DisMaxRequestHandlerTest, TestPerFieldSimilarity, AsyncMigrateRouteKeyTest, TestCollectionAPI, TestSolrQueryParser, ExternalCollectionsTest, ClusterStateTest, SampleTest, HdfsDirectoryTest, TestCopyFieldCollectionResource, SyncSliceTest, ZkCLITest, SuggesterTest, ExitableDirectoryReaderTest, BasicDistributedZk2Test, TestHighlightDedupGrouping, TestSortingResponseWriter, PathHierarchyTokenizerFactoryTest, ChaosMonkeyNothingIsSafeTest, DocValuesTest, TestAtomicUpdateErrorCases, FacetPivotSmallTest, TestQuerySenderNoQuery, CollectionsAPIDistributedZkTest, LeaderElectionIntegrationTest, TestConfigOverlay, TestConfigSets, AssignTest, SOLR749Test, ZkSolrClientTest, TestManagedSchemaFieldResource, TestWordDelimiterFilterFactory, StatsComponentTest, SimpleMLTQParserTest, EchoParamsTest, TestFiltering, TestFieldTypeCollectionResource, OverseerCollectionProcessorTest, TestComponentsName, TestMergePolicyConfig, UpdateParamsTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestNRTOpen, HighlighterTest, TestLMDirichletSimilarityFactory, TestInfoStreamLogging, SignatureUpdateProcessorFactoryTest, TestZkChroot, DocumentBuilderTest, TestBulkSchemaConcurrent, HdfsWriteToMultipleCollectionsTest, TestShortCircuitedRequests, PluginInfoTest, HighlighterConfigTest, TestReplicaProperties, TestManagedResource, OutputWriterTest, UpdateRequestProcessorFactoryTest, TestSchemaVersionResource, SimpleFacetsTest, FieldAnalysisRequestHandlerTest, TestDocBasedVersionConstraints, TestCSVResponseWriter, SearchHandlerTest, NoCacheHeaderTest, TestNonNRTOpen, TestFaceting, JsonLoaderTest, SuggesterWFSTTest, ClusterStateUpdateTest, TestRequestStatusCollectionAPI, TestConfig, TestCoreDiscovery, TestBlendedInfixSuggestions, LoggingHandlerTest, TestFieldTypeResource, TestTolerantSearch, DistributedQueryElevationComponentTest, DirectSolrSpellCheckerTest, SSLMigrationTest, SpellCheckComponentTest, RegexBoostProcessorTest, CurrencyFieldOpenExchangeTest, ZkNodePropsTest, TestStressLucene, AddBlockUpdateTest, TestComplexPhraseQParserPlugin, HdfsRecoveryZkTest, TestRandomMergePolicy, TestUtils, TimeZoneUtilsTest, HdfsLockFactoryTest, SolrPluginUtilsTest, CoreAdminCreateDiscoverTest, SharedFSAutoReplicaFailoverUtilsTest, SystemInfoHandlerTest, TestSolrQueryParserResource, BasicFunctionalityTest, TestRestManager, BasicDistributedZkTest, UnloadDistributedZkTest, OpenCloseCoreStressTest, OverseerTest, ShardRoutingTest, BasicZkTest, RecoveryZkTest, TestReplicationHandler, TestRandomFaceting, LeaderElectionTest, TestRandomDVFaceting, AutoCommitTest, TestDistributedGrouping, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, TestStressReorder, TestJoin, TestMultiCoreConfBootstrap, TestReload, DistributedTermsComponentTest, TestRangeQuery, TestSolr4Spatial, TestGroupingSearch, SolrCmdDistributorTest, QueryElevationComponentTest, TestSort, TestFunctionQuery, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, TestIndexSearcher, SoftAutoCommitTest, ShowFileRequestHandlerTest, SolrIndexSplitterTest, SimplePostToolTest, SpellCheckCollatorTest, SuggesterFSTTest, SolrRequestParserTest, TestFoldingMultitermQuery, TestCSVLoader, PolyFieldTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DocValuesMultiTest, DistanceFunctionTest, TestSolrDeletionPolicy1, SolrInfoMBeanTest, XsltUpdateRequestHandlerTest, DebugComponentTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, TermsComponentTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, TestValueSourceCache, MoreLikeThisHandlerTest, RequiredFieldsTest, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, IndexSchemaTest, TestJmxIntegration, TestCollationField, ReturnFieldsTest, JSONWriterTest, QueryParsingTest, TestAnalyzedSuggestions, TestPartialUpdateDeduplication, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, TestBinaryResponseWriter, TestLFUCache, AlternateDirectoryTest, TestSolrIndexConfig, CopyFieldTest, BadComponentTest, TestDocSet, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, TestElisionMultitermQuery, TestFuzzyAnalyzedSuggestions, TestPostingsSolrHighlighter, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestPhraseSuggestions, TestCharFilters, SynonymTokenizerTest, TestDFRSimilarityFactory, TestIBSimilarityFactory, TestDefaultSimilarityFactory, TestFastWriter, LegacyHTMLStripCharFilterTest, TestJmxMonitoredMap, URLClassifyProcessorTest, TestFastLRUCache, PreAnalyzedFieldTest, TestSuggestSpellingConverter, DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, TestDocumentBuilder, UUIDFieldTest, FileUtilsTest, CircularListTest, CursorPagingTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, AliasIntegrationTest, CollectionsAPIAsyncDistributedZkTest, DeleteReplicaTest, DeleteShardTest, DistribCursorPagingTest, DistribDocExpirationUpdateProcessorTest, DistributedQueueTest, HttpPartitionTest, LeaderInitiatedRecoveryOnCommitTest, MigrateRouteKeyTest, MultiThreadedOCPTest, OverriddenZkACLAndCredentialsProvidersTest, ReplicationFactorTest, RollingRestartTest, SharedFSAutoReplicaFailoverTest, SimpleCollectionCreateDeleteTest, SolrXmlInZkTest, TestCloudInspectUtil, TestCloudPivotFacet, TestDistribDocBasedVersion, TestLeaderElectionZkExpiry, VMParamsZkACLAndCredentialsProvidersTest, HdfsBasicDistributedZk2Test, HdfsChaosMonkeySafeLeaderTest, HdfsCollectionsAPIDistributedZkTest, HdfsSyncSliceTest, StressHdfsTest, TestClusterStateMutator, CachingDirectoryFactoryTest, QueryResultKeyTest, TestInitParams, TestReloadAndDeleteDocs, TestShardHandlerFactory, TestSolrDynamicMBean, TestSolrXml, TestSolrXmlPersistence, TestReplicationHandlerBackup]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandlerBackup -Dtests.seed=4F62D3D25588F392 -Dtests.slow=true -Dtests.locale=es_MX -Dtests.timezone=HST -Dtests.asserts=false -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s | TestReplicationHandlerBackup (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)
   [junit4] Completed in 7221.83s, 1 test, 2 errors <<< FAILURES!

[...truncated 195 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:519: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:467: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:510: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1349: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:956: There were test failures: 451 suites, 1840 tests, 3 suite-level errors, 1 error, 56 ignored (31 assumptions)

Total time: 257 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.8.0_20 -server -XX:+UseParallelGC (asserts: false)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_40-ea-b09) - Build # 4463 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4463/
Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseParallelGC (asserts: false)

2 tests failed.
FAILED:  org.apache.solr.client.solrj.embedded.JettyWebappTest.testAdminUI

Error Message:
Server returned HTTP response code: 503 for URL: http://127.0.0.1:55057/test/

Stack Trace:
java.io.IOException: Server returned HTTP response code: 503 for URL: http://127.0.0.1:55057/test/
	at __randomizedtesting.SeedInfo.seed([614C5E38F26C5E76:599EBD6B9507C5C9]:0)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1838)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
	at java.net.URL.openStream(URL.java:1038)
	at org.apache.solr.client.solrj.embedded.JettyWebappTest.testAdminUI(JettyWebappTest.java:102)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.client.solrj.embedded.JettyWebappTest

Error Message:
7 threads leaked from SUITE scope at org.apache.solr.client.solrj.embedded.JettyWebappTest:     1) Thread[id=113, name=qtp1117973912-113, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=114, name=qtp1117973912-114, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=112, name=qtp1117973912-112, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=116, name=qtp1117973912-116, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=111, name=qtp1117973912-111, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=115, name=qtp1117973912-115, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=117, name=qtp1117973912-117, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.client.solrj.embedded.JettyWebappTest: 
   1) Thread[id=113, name=qtp1117973912-113, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=114, name=qtp1117973912-114, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=112, name=qtp1117973912-112, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=116, name=qtp1117973912-116, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=111, name=qtp1117973912-111, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=115, name=qtp1117973912-115, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=117, name=qtp1117973912-117, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([614C5E38F26C5E76]:0)




Build Log:
[...truncated 10478 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.embedded.JettyWebappTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.JettyWebappTest-614C5E38F26C5E76-001\init-core-data-001
   [junit4]   2> 33958 T109 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 33960 T109 oas.SolrTestCaseJ4.setUp ###Starting testAdminUI
   [junit4]   2> 34383 T109 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 34613 T109 oejw.StandardDescriptorProcessor.visitServlet NO JSP Support for /test, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 34650 T109 oejw.WebAppContext.doStart WARN Failed startup of context o.e.j.w.WebAppContext{/test,file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/webapp/web/},C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\webapp\web java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	at org.eclipse.jetty.webapp.IterativeDescriptorProcessor.visit(IterativeDescriptorProcessor.java:85)
   [junit4]   2> 	at org.eclipse.jetty.webapp.IterativeDescriptorProcessor.process(IterativeDescriptorProcessor.java:72)
   [junit4]   2> 	at org.eclipse.jetty.webapp.MetaData.resolve(MetaData.java:367)
   [junit4]   2> 	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:710)
   [junit4]   2> 	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:494)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:95)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.doStart(Server.java:280)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettyWebappTest.setUp(JettyWebappTest.java:81)
   [junit4]   2> 	at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:861)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.IllegalStateException: No such servlet: SolrConfigRestApi
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1274)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.setServletMappings(ServletHandler.java:1368)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.addServletMapping(ServletHandler.java:851)
   [junit4]   2> 	at org.eclipse.jetty.webapp.StandardDescriptorProcessor.addServletMapping(StandardDescriptorProcessor.java:1198)
   [junit4]   2> 	at org.eclipse.jetty.webapp.StandardDescriptorProcessor.visitServletMapping(StandardDescriptorProcessor.java:632)
   [junit4]   2> 	... 53 more
   [junit4]   2> 
   [junit4]   2> 34660 T109 oejs.AbstractConnector.doStart Started SocketConnector@0.0.0.0:55057
   [junit4]   2> 34763 T109 oejsh.ContextHandler.doStop stopped o.e.j.w.WebAppContext{/test,file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/webapp/web/},C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\webapp\web
   [junit4]   2> 34766 T109 oejuc.AbstractLifeCycle.setFailed WARN FAILED o.e.j.w.WebAppContext{/test,file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/webapp/web/},C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\webapp\web: java.lang.IllegalStateException: No such servlet: SolrConfigRestApi java.lang.IllegalStateException: No such servlet: SolrConfigRestApi
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1274)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1345)
   [junit4]   2> 	at org.eclipse.jetty.webapp.WebXmlConfiguration.deconfigure(WebXmlConfiguration.java:129)
   [junit4]   2> 	at org.eclipse.jetty.webapp.WebAppContext.doStop(WebAppContext.java:523)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.doStop(HandlerWrapper.java:107)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.doStop(Server.java:341)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettyWebappTest.tearDown(JettyWebappTest.java:89)
   [junit4]   2> 	at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 34767 T109 oejuc.AbstractLifeCycle.setFailed WARN FAILED org.eclipse.jetty.server.Server@77de7979: java.lang.IllegalStateException: No such servlet: SolrConfigRestApi java.lang.IllegalStateException: No such servlet: SolrConfigRestApi
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1274)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1345)
   [junit4]   2> 	at org.eclipse.jetty.webapp.WebXmlConfiguration.deconfigure(WebXmlConfiguration.java:129)
   [junit4]   2> 	at org.eclipse.jetty.webapp.WebAppContext.doStop(WebAppContext.java:523)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.doStop(HandlerWrapper.java:107)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.doStop(Server.java:341)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettyWebappTest.tearDown(JettyWebappTest.java:89)
   [junit4]   2> 	at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 34767 T109 oas.SolrTestCaseJ4.tearDown ###Ending testAdminUI
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=JettyWebappTest -Dtests.method=testAdminUI -Dtests.seed=614C5E38F26C5E76 -Dtests.slow=true -Dtests.locale=sv_SE -Dtests.timezone=America/Indianapolis -Dtests.asserts=false -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.89s | JettyWebappTest.testAdminUI <<<
   [junit4]    > Throwable #1: java.io.IOException: Server returned HTTP response code: 503 for URL: http://127.0.0.1:55057/test/
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([614C5E38F26C5E76:599EBD6B9507C5C9]:0)
   [junit4]    > 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1838)
   [junit4]    > 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
   [junit4]    > 	at java.net.URL.openStream(URL.java:1038)
   [junit4]    > 	at org.apache.solr.client.solrj.embedded.JettyWebappTest.testAdminUI(JettyWebappTest.java:102)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 34845 T109 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.JettyWebappTest-614C5E38F26C5E76-001
   [junit4]   2> 892 T108 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 7 leaked thread(s).
   [junit4]   2> 21025 T108 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 7 threads leaked from SUITE scope at org.apache.solr.client.solrj.embedded.JettyWebappTest: 
   [junit4]   2> 	   1) Thread[id=113, name=qtp1117973912-113, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   2) Thread[id=114, name=qtp1117973912-114, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   3) Thread[id=112, name=qtp1117973912-112, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   4) Thread[id=116, name=qtp1117973912-116, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   5) Thread[id=111, name=qtp1117973912-111, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   6) Thread[id=115, name=qtp1117973912-115, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   7) Thread[id=117, name=qtp1117973912-117, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 21026 T108 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=113, name=qtp1117973912-113, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	   2) Thread[id=114, name=qtp1117973912-114, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	   3) Thread[id=112, name=qtp1117973912-112, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	   4) Thread[id=116, name=qtp1117973912-116, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	   5) Thread[id=111, name=qtp1117973912-111, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	   6) Thread[id=115, name=qtp1117973912-115, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	   7) Thread[id=117, name=qtp1117973912-117, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 21028 T108 ccr.ThreadLeakControl.tryToInterruptAll All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=sv_SE, timezone=America/Indianapolis
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_40-ea (64-bit)/cpus=2,threads=1,free=100345544,total=207093760
   [junit4]   2> NOTE: All tests run in this JVM: [ExternalHttpClientTest, ContentStreamTest, TestDocumentObjectBinder, LargeVolumeEmbeddedTest, SolrExampleBinaryTest, TestNamedListCodec, TestRetryUtil, TermsResponseTest, ModifiableSolrParamsTest, MergeIndexesEmbeddedTest, MultiCoreEmbeddedTest, TestHash, DocumentAnalysisResponseTest, SolrDocumentTest, JettyWebappTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=JettyWebappTest -Dtests.seed=614C5E38F26C5E76 -Dtests.slow=true -Dtests.locale=sv_SE -Dtests.timezone=America/Indianapolis -Dtests.asserts=false -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s | JettyWebappTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.client.solrj.embedded.JettyWebappTest: 
   [junit4]    >    1) Thread[id=113, name=qtp1117973912-113, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=114, name=qtp1117973912-114, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    3) Thread[id=112, name=qtp1117973912-112, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    4) Thread[id=116, name=qtp1117973912-116, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    5) Thread[id=111, name=qtp1117973912-111, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    6) Thread[id=115, name=qtp1117973912-115, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    7) Thread[id=117, name=qtp1117973912-117, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([614C5E38F26C5E76]:0)
   [junit4] Completed in 21.06s, 1 test, 2 errors <<< FAILURES!

[...truncated 128 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:519: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:467: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:202: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:510: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1349: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:956: There were test failures: 54 suites, 330 tests, 1 suite-level error, 1 error

Total time: 145 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseParallelGC (asserts: false)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_20) - Build # 4462 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4462/
Java: 64bit/jdk1.8.0_20 -XX:-UseCompressedOops -XX:+UseG1GC (asserts: false)

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderInitiatedRecoveryOnCommitTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderInitiatedRecoveryOnCommitTest:     1) Thread[id=2541, name=SessionTracker, state=TIMED_WAITING, group=TGRP-LeaderInitiatedRecoveryOnCommitTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderInitiatedRecoveryOnCommitTest: 
   1) Thread[id=2541, name=SessionTracker, state=TIMED_WAITING, group=TGRP-LeaderInitiatedRecoveryOnCommitTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
	at __randomizedtesting.SeedInfo.seed([A04E2CBCA5501A09]:0)


FAILED:  org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([A04E2CBCA5501A09]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([A04E2CBCA5501A09]:0)




Build Log:
[...truncated 8898 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandlerBackup
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\init-core-data-001
   [junit4]   2> 518677 T1322 oas.SolrTestCaseJ4.setUp ###Starting doTestBackup
   [junit4]   2> 518688 T1322 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 518713 T1322 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58429
   [junit4]   2> 518714 T1322 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 518714 T1322 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 518715 T1322 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001
   [junit4]   2> 518715 T1322 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\'
   [junit4]   2> 518744 T1322 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\solr.xml
   [junit4]   2> 518755 T1322 oasc.CoreContainer.<init> New CoreContainer 1797947065
   [junit4]   2> 518755 T1322 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\]
   [junit4]   2> 518755 T1322 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 518756 T1322 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 518756 T1322 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 518756 T1322 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 518757 T1322 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 518757 T1322 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 518757 T1322 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 518757 T1322 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 518757 T1322 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 518757 T1322 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 518758 T1322 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 518758 T1322 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 518760 T1322 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 518760 T1322 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 518766 T1332 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\'
   [junit4]   2> 518788 T1332 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 518808 T1332 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 518810 T1332 oass.IndexSchema.readSchema Reading Solr Schema from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\conf\schema.xml
   [junit4]   2> 518811 T1332 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 518813 T1332 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 518813 T1332 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\
   [junit4]   2> 518813 T1332 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 518814 T1332 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\data\
   [junit4]   2> 518814 T1332 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 518814 T1332 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\data\
   [junit4]   2> 518814 T1332 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\data\index/
   [junit4]   2> 518814 T1332 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\data\index' doesn't exist. Creating new index...
   [junit4]   2> 518814 T1332 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\data\index
   [junit4]   2> 518814 T1332 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=81.30859375, floorSegmentMB=1.8857421875, forceMergeDeletesPctAllowed=2.4175444507422474, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 518815 T1332 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@68a2f7b2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6dc123f7),segFN=segments_1,generation=1}
   [junit4]   2> 518815 T1332 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 518816 T1332 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 518817 T1332 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 518817 T1332 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 518817 T1332 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 518817 T1332 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 518817 T1332 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 518817 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 518818 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 518818 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 518818 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 518819 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 518819 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 518819 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 518819 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: org.apache.solr.handler.admin.ShowFileRequestHandler
   [junit4]   2> 518819 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 518819 T1332 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 518832 T1332 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 518832 T1332 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 518832 T1332 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 518832 T1332 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 518833 T1332 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 518837 T1332 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 518839 T1332 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 518839 T1332 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 518841 T1332 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 518841 T1332 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 518841 T1332 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 518842 T1332 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=42, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 518842 T1332 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@68a2f7b2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6dc123f7),segFN=segments_1,generation=1}
   [junit4]   2> 518843 T1332 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 518843 T1332 oass.SolrIndexSearcher.<init> Opening Searcher@6ad4d6d3[collection1] main
   [junit4]   2> 518843 T1332 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\conf
   [junit4]   2> 518843 T1332 oasr.RestManager.init Initializing RestManager with initArgs: {storageDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\conf}
   [junit4]   2> 518843 T1332 oasr.ManagedResourceStorage.load Reading _rest_managed.json using file:dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\conf
   [junit4]   2> 518844 T1332 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 518845 T1332 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using file:dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\conf
   [junit4]   2> 518845 T1332 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 518845 T1332 oash.ReplicationHandler.inform Replication enabled for following config files: schema-replication2.xml:schema.xml
   [junit4]   2> 518845 T1332 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 518845 T1332 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 518845 T1333 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6ad4d6d3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 518846 T1322 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 518846 T1322 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C200 name=collection1 org.apache.solr.core.SolrCore@4e47c1fe
   [junit4]   2> 518851 T1326 C200 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 518853 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 4
   [junit4]   2> 518857 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 1
   [junit4]   2> 518860 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 0
   [junit4]   2> 518862 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 0
   [junit4]   2> 518864 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[3]} 0 0
   [junit4]   2> 518866 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[4]} 0 0
   [junit4]   2> 518868 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5]} 0 0
   [junit4]   2> 518871 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6]} 0 1
   [junit4]   2> 518873 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7]} 0 0
   [junit4]   2> 518875 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8]} 0 0
   [junit4]   2> 518877 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[9]} 0 0
   [junit4]   2> 518879 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[10]} 0 0
   [junit4]   2> 518881 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[11]} 0 0
   [junit4]   2> 518883 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[12]} 0 0
   [junit4]   2> 518885 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[13]} 0 0
   [junit4]   2> 518887 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[14]} 0 0
   [junit4]   2> 518889 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[15]} 0 0
   [junit4]   2> 518891 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[16]} 0 0
   [junit4]   2> 518893 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[17]} 0 0
   [junit4]   2> 518896 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[18]} 0 0
   [junit4]   2> 518898 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[19]} 0 0
   [junit4]   2> 518900 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[20]} 0 0
   [junit4]   2> 518902 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[21]} 0 0
   [junit4]   2> 518904 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[22]} 0 0
   [junit4]   2> 518907 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[23]} 0 0
   [junit4]   2> 518909 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[24]} 0 0
   [junit4]   2> 518911 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[25]} 0 0
   [junit4]   2> 518913 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[26]} 0 0
   [junit4]   2> 518915 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[27]} 0 0
   [junit4]   2> 518917 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[28]} 0 0
   [junit4]   2> 518919 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[29]} 0 0
   [junit4]   2> 518921 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[30]} 0 0
   [junit4]   2> 518923 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[31]} 0 0
   [junit4]   2> 518925 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[32]} 0 0
   [junit4]   2> 518926 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[33]} 0 0
   [junit4]   2> 518928 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[34]} 0 0
   [junit4]   2> 518931 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[35]} 0 0
   [junit4]   2> 518932 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[36]} 0 0
   [junit4]   2> 518933 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[37]} 0 0
   [junit4]   2> 518935 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[38]} 0 0
   [junit4]   2> 518936 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[39]} 0 0
   [junit4]   2> 518938 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[40]} 0 0
   [junit4]   2> 518939 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[41]} 0 0
   [junit4]   2> 518942 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[42]} 0 0
   [junit4]   2> 518944 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[43]} 0 0
   [junit4]   2> 518946 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[44]} 0 0
   [junit4]   2> 518948 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[45]} 0 0
   [junit4]   2> 518950 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[46]} 0 0
   [junit4]   2> 518952 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[47]} 0 0
   [junit4]   2> 518954 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[48]} 0 0
   [junit4]   2> 518956 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[49]} 0 0
   [junit4]   2> 518958 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[50]} 0 0
   [junit4]   2> 518960 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[51]} 0 0
   [junit4]   2> 518963 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[52]} 0 0
   [junit4]   2> 518965 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[53]} 0 0
   [junit4]   2> 518967 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[54]} 0 0
   [junit4]   2> 518970 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[55]} 0 0
   [junit4]   2> 518972 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[56]} 0 0
   [junit4]   2> 518974 T1330 C200 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 518977 T1330 C200 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@68a2f7b2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6dc123f7),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@68a2f7b2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6dc123f7),segFN=segments_2,generation=2}
   [junit4]   2> 518978 T1330 C200 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 518979 T1330 C200 oass.SolrIndexSearcher.<init> Opening Searcher@2335b24[collection1] main
   [junit4]   2> 518979 T1330 C200 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 518979 T1333 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2335b24[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.0.0):C57)))}
   [junit4]   2> 518979 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 5
   [junit4]   2> 519040 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=aeavo&command=backup} status=0 QTime=9 
   [junit4]   2> 519047 T1335 oash.SnapShooter.createSnapshot Creating backup snapshot...
   [junit4]   2> 519057 T1335 oash.SnapShooter.createSnapshot Done creating backup snapshot: aeavo
   [junit4]   2> 519058 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=5 
   [junit4]   2> 519200 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 519914 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=gapqmz&command=backup} status=0 QTime=1 
   [junit4]   2> 519915 T1337 oash.SnapShooter.createSnapshot Creating backup snapshot...
   [junit4]   2> 519922 T1337 oash.SnapShooter.createSnapshot Done creating backup snapshot: gapqmz
   [junit4]   2> 519922 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=5 
   [junit4]   2> 520065 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 520074 T1338 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: aeavo
   [junit4]   2> 520075 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=aeavo&command=deletebackup} status=0 QTime=4 
   [junit4]   2> 520079 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 520790 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=gapqmz&command=deletebackup} status=0 QTime=1 
   [junit4]   2> 520790 T1339 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: gapqmz
   [junit4]   2> 520794 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 520937 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 521079 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 521222 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 521364 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 521506 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 521658 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 521816 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 521974 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 522131 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 522288 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 522444 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 522601 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 522758 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 522915 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 523072 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 523230 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 523388 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 523545 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 523702 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 523858 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 524015 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 524173 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 524330 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 524488 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 524645 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 524807 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 524964 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 525121 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 525278 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 525436 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 525593 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 525751 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 525908 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 526066 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 526223 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 526380 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 526537 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 526693 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 526851 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 527008 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 527164 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 527320 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 527477 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 527634 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 527792 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 527950 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 528105 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 528262 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 528418 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 528574 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 528731 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 528888 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 529044 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 529201 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 529373 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 529545 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 529717 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 529889 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 530061 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 530234 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 530406 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 530578 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 530750 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 530922 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 531093 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 531266 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 531438 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 531610 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 531782 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 531954 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 532126 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 532298 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 532470 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 532642 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 532814 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 532986 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 533158 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 533330 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 533502 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 533675 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 533847 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 534019 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 534191 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 534363 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 534535 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 534706 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 534879 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 535052 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 535224 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 535395 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 535568 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 535740 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 535912 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 536084 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 536256 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 536428 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 536600 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 536772 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 536944 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 537116 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 537288 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 537460 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 537632 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 537804 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 537992 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 538182 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 538371 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 538559 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 538749 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 538940 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 539130 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 539320 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 539508 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 539697 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 539886 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 540076 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 540265 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 540454 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 540643 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 540833 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 541021 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 541210 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 541400 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 541589 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 541778 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 541967 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 542156 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 542345 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 542537 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 542726 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 542915 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 543105 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 543294 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 543482 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 543670 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 543860 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 544050 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 544239 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 544429 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 544618 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 544807 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 544996 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 545186 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 545375 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 545564 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 545753 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 545942 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 546132 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 546321 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 546510 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 546699 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 546888 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 547078 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 547277 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 547485 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 547692 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 547900 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 548107 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 548314 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 548522 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 548731 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 548941 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 549151 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 549359 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 549566 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 549774 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 549983 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 550191 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 550399 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 550608 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 550816 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 551024 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 551232 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 551440 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 551648 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 551856 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 552064 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 552272 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 552481 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 552689 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 552897 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 553105 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 553313 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 553520 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 553729 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 553936 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 554146 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 554353 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 554562 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 554770 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 554978 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 555186 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 555394 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 555601 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 555810 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 556020 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 556228 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 556435 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 556644 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 556852 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 557060 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 557268 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 557478 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 557708 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 557937 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 558165 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 558396 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 558624 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 558853 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 559082 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 559311 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 559545 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 559777 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 560006 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 560235 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 560465 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 560694 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 560924 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 561154 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 561383 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 561613 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 561842 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 562072 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 562300 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 562529 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 562758 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 562987 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 563215 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 563445 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 563674 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 563902 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 564132 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 564360 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 564590 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 564819 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 565048 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 565278 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 565508 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 565738 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 565967 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 566197 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 566437 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 566665 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 566895 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 567126 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 567358 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 567590 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 567828 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 568058 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 568289 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 568524 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 568754 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 569004 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 569258 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 569510 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 569762 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 570014 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 570266 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 570517 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 570769 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 571021 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 571273 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 571526 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 571778 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 572030 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 572283 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 572536 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 572787 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 573039 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 573291 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 573543 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 573795 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 574046 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 574298 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 574549 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 574801 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 575051 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 575303 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 575556 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 575808 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 576060 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 576312 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 576583 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 576835 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 577087 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 577338 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 577590 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 577841 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 578093 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 578344 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 578596 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 578849 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 579101 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 579352 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 579605 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 579856 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 580108 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 580360 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 580612 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 580864 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 581116 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 581378 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 581657 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 581934 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 582211 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 582487 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 582764 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 583040 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 583317 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 583594 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 583872 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 584148 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 584425 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 584703 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 584980 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 585256 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 585533 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 585810 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 586087 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 586364 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 586641 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 586917 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 587194 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 587471 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 587749 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 588026 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 588302 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 588577 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 588854 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 589130 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 589407 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 589681 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 589958 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 590234 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 590511 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 590788 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 591065 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 591340 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 591617 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 591894 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 592171 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 592448 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 592725 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 593001 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 593277 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 593554 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 593829 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 594106 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 594382 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 594658 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 594934 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 595235 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 595540 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 595843 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 596146 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 596449 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 596753 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 597056 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 597359 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 597662 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 597965 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 598270 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 598573 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 598876 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 599181 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 599484 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 599788 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 600091 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 600394 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 600697 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 601000 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 601304 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 601608 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 601912 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 602215 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 602518 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 602821 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 603123 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 603426 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 603729 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 604033 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 604336 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 604639 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 604942 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 605245 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 605547 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 605850 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 606152 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 606455 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 606757 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 607060 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 607364 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 607667 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 607970 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 608272 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 608575 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 608878 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 609181 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 609485 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 609786 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 610090 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 610391 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 610693 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 610998 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 611301 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 611604 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 611907 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 612212 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 612515 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 612818 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 613122 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 613425 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 613728 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 614030 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 614333 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 614635 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 614938 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 615241 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 615544 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 615846 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 616149 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 616451 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 616756 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 617059 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status

[...truncated too long message...]

eakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 7657696 T2538 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 7657696 T2538 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderInitiatedRecoveryOnCommitTest-A04E2CBCA5501A09-001\tempDir-001/jetty4\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderInitiatedRecoveryOnCommitTest-A04E2CBCA5501A09-001\tempDir-001/jetty4\index;done=false>>]
   [junit4]   2> 7657696 T2538 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderInitiatedRecoveryOnCommitTest-A04E2CBCA5501A09-001\tempDir-001/jetty4\index
   [junit4]   2> 7657696 T2538 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderInitiatedRecoveryOnCommitTest-A04E2CBCA5501A09-001\tempDir-001/jetty4\ [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderInitiatedRecoveryOnCommitTest-A04E2CBCA5501A09-001\tempDir-001/jetty4\;done=false>>]
   [junit4]   2> 7657697 T2538 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderInitiatedRecoveryOnCommitTest-A04E2CBCA5501A09-001\tempDir-001/jetty4\
   [junit4]   2> 7657698 T2538 oasc.Overseer.close Overseer (id=92890117425856524-127.0.0.1:59056_-n_0000000004) closing
   [junit4]   2> 7657701 T2834 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:59056_
   [junit4]   2> 7659849 T2832 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 7659875 T2538 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 7667182 T2538 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 59005
   [junit4]   2> 7667182 T2538 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 7667185 T2538 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:58998 58998
   [junit4]   2> 7670486 T2539 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:58998 58998
   [junit4]   2> 7670493 T2539 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations: 
   [junit4]   2> 	Maximum concurrent create/delete watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		6	/solr/aliases.json
   [junit4]   2> 		6	/solr/clusterstate.json
   [junit4]   2> 		5	/solr/configs/conf1
   [junit4]   2> 	
   [junit4]   2> 	Maximum concurrent children watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		6	/solr/live_nodes
   [junit4]   2> 		5	/solr/overseer/collection-queue-work
   [junit4]   2> 		4	/solr/overseer/queue
   [junit4]   2> 	
   [junit4]   2> 7670494 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59040/
   [junit4]   2> 7670494 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59056/
   [junit4]   2> 7670496 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59005/
   [junit4]   2> 7670497 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59028/
   [junit4]   2> 7670497 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59018/
   [junit4]   2> 7670499 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59040/
   [junit4]   2> 7670500 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59056/
   [junit4]   2> 7670500 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59005/
   [junit4]   2> 7670500 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59028/
   [junit4]   2> 7670500 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59018/
   [junit4]   2> 7670502 T2538 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> -508974 T2537 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> -488610 T2537 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderInitiatedRecoveryOnCommitTest: 
   [junit4]   2> 	   1) Thread[id=2541, name=SessionTracker, state=TIMED_WAITING, group=TGRP-LeaderInitiatedRecoveryOnCommitTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
   [junit4]   2> -488610 T2537 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=2541, name=SessionTracker, state=TIMED_WAITING, group=TGRP-LeaderInitiatedRecoveryOnCommitTest]
   [junit4]   2> 7694040 T2541 oazs.SessionTrackerImpl.run ERROR Unexpected interruption java.lang.InterruptedException
   [junit4]   2> 	at java.lang.Object.wait(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
   [junit4]   2> 
   [junit4]   2> -488606 T2537 ccr.ThreadLeakControl.tryToInterruptAll All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50), sim=DefaultSimilarity, locale=uk, timezone=Pacific/Easter
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_20 (64-bit)/cpus=2,threads=1,free=88045024,total=381681664
   [junit4]   2> NOTE: All tests run in this JVM: [BlockDirectoryTest, SchemaVersionSpecificBehaviorTest, ResourceLoaderTest, TestCharFilters, CoreMergeIndexesAdminHandlerTest, TestCollationField, LukeRequestHandlerTest, TestMissingGroups, OutputWriterTest, FileUtilsTest, TestPivotHelperCode, MultiTermTest, RAMDirectoryFactoryTest, SuggestComponentTest, TestLuceneMatchVersion, SyncSliceTest, TestAnalyzeInfixSuggestions, DistanceFunctionTest, TestNonNRTOpen, TestStressUserVersions, PreAnalyzedUpdateProcessorTest, TestGroupingSearch, ChangedSchemaMergeTest, TestPartialUpdateDeduplication, TestCoreContainer, ChaosMonkeyNothingIsSafeTest, SpellingQueryConverterTest, HttpPartitionTest, BadCopyFieldTest, AddBlockUpdateTest, TestCollectionAPI, TestDynamicFieldResource, BasicFunctionalityTest, TestSolrXml, BJQParserTest, ResponseHeaderTest, TestCustomSort, CursorPagingTest, ZkControllerTest, TestPseudoReturnFields, TestSolrDeletionPolicy1, ResponseLogComponentTest, DistributedExpandComponentTest, TestReplicationHandlerBackup, TestFastWriter, ExternalCollectionsTest, DistributedDebugComponentTest, MinimalSchemaTest, TestLFUCache, TestStressReorder, LeaderElectionTest, CurrencyFieldOpenExchangeTest, SpellCheckComponentTest, SolrCmdDistributorTest, TestUniqueKeyFieldResource, DebugComponentTest, TestSimpleQParserPlugin, OverseerRolesTest, PeerSyncTest, TestSchemaVersionResource, DateMathParserTest, MigrateRouteKeyTest, TestIBSimilarityFactory, ParsingFieldUpdateProcessorsTest, TestSolrXmlPersistence, TestConfigReload, TestSchemaResource, LeaderInitiatedRecoveryOnCommitTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnCommitTest -Dtests.seed=A04E2CBCA5501A09 -Dtests.slow=true -Dtests.locale=uk -Dtests.timezone=Pacific/Easter -Dtests.asserts=false -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s | LeaderInitiatedRecoveryOnCommitTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderInitiatedRecoveryOnCommitTest: 
   [junit4]    >    1) Thread[id=2541, name=SessionTracker, state=TIMED_WAITING, group=TGRP-LeaderInitiatedRecoveryOnCommitTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A04E2CBCA5501A09]:0)
   [junit4] Completed in -488.55s, 1 test, 1 error <<< FAILURES!

[...truncated 1237 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:519: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:467: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:510: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1349: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:956: There were test failures: 451 suites, 1840 tests, 2 suite-level errors, 1 error, 56 ignored (31 assumptions)

Total time: 270 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0_20 -XX:-UseCompressedOops -XX:+UseG1GC (asserts: false)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any