You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/09/14 02:48:01 UTC

[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) - Build # 4310 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4310/
Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.benchmark.byTask.TestPerfTasksParse

Error Message:
Could not remove the following files (in the order of attempts):    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of attempts):
   C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
   C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
   C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001

	at __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0)
	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288)
	at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:126)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 7622 lines...]
   [junit4] Suite: org.apache.lucene.benchmark.byTask.TestPerfTasksParse
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> Spatial Strategy: RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:26,ctx:SpatialContext{geo=true, calculator=Haversine, worldBounds=Rect(minX=-180.0,maxX=180.0,minY=-90.0,maxY=90.0)})),pointsOnly,pruneLeafyBranches)
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=DefaultSimilarity, locale=sq_AL, timezone=America/Yakutat
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_67 (64-bit)/cpus=2,threads=1,free=120281832,total=140083200
   [junit4]   2> NOTE: All tests run in this JVM: [DocMakerTest, WriteLineDocTaskTest, AddIndexesTaskTest, StreamUtilsTest, TrecContentSourceTest, TestQualityRun, PerfTaskTest, LineDocSourceTest, TestPerfTasksLogic, TestPerfTasksParse]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestPerfTasksParse -Dtests.seed=2D34950549928D3F -Dtests.slow=true -Dtests.locale=sq_AL -Dtests.timezone=America/Yakutat -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s | TestPerfTasksParse (suite) <<<
   [junit4]    > Throwable #1: java.io.IOException: Could not remove the following files (in the order of attempts):
   [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
   [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
   [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-2D34950549928D3F-001
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed in 3.00s, 3 tests, 1 error <<< FAILURES!

[...truncated 33 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:491: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:471: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:60: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build.xml:462: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:2140: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\module-build.xml:58: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1358: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:961: There were test failures: 18 suites, 93 tests, 1 suite-level error

Total time: 31 minutes 50 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.7.0_67) - Build # 4313 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4313/
Java: 32bit/jdk1.7.0_67 -client -XX:+UseSerialGC

3 tests failed.
REGRESSION:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testDistribSearch

Error Message:
Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1
	at __randomizedtesting.SeedInfo.seed([40015FF1CB47EE66:C1E7D1E9BC188E5A]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:583)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


REGRESSION:  org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.testDistribSearch

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:55043/rpswv/k, http://127.0.0.1:55019/rpswv/k, http://127.0.0.1:55034/rpswv/k]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:55043/rpswv/k, http://127.0.0.1:55019/rpswv/k, http://127.0.0.1:55034/rpswv/k]
	at __randomizedtesting.SeedInfo.seed([40015FF1CB47EE66:C1E7D1E9BC188E5A]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:322)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:874)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
	at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.removeAndWaitForLastReplicaGone(DeleteLastCustomShardedReplicaTest.java:116)
	at org.apache.solr.cloud.DeleteLastCustomShardedReplicaTest.doTest(DeleteLastCustomShardedReplicaTest.java:106)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.GeneratedMethodAccessor60.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not  remove replica : customcollreplicadeletion/a/core_node1
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304)
	... 45 more


REGRESSION:  org.apache.solr.cloud.DeleteReplicaTest.testDistribSearch

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:64255, http://127.0.0.1:64273, http://127.0.0.1:64282, http://127.0.0.1:64240, http://127.0.0.1:64264]

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:64255, http://127.0.0.1:64273, http://127.0.0.1:64282, http://127.0.0.1:64240, http://127.0.0.1:64264]
	at __randomizedtesting.SeedInfo.seed([40015FF1CB47EE66:C1E7D1E9BC188E5A]:0)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:322)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.sendRequest(CloudSolrServer.java:874)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.requestWithRetryOnStaleState(CloudSolrServer.java:658)
	at org.apache.solr.client.solrj.impl.CloudSolrServer.request(CloudSolrServer.java:601)
	at org.apache.solr.cloud.DeleteReplicaTest.removeAndWaitForReplicaGone(DeleteReplicaTest.java:139)
	at org.apache.solr.cloud.DeleteReplicaTest.deleteLiveReplicaTest(DeleteReplicaTest.java:123)
	at org.apache.solr.cloud.DeleteReplicaTest.doTest(DeleteReplicaTest.java:87)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Could not  remove replica : delLiveColl/shard2/core_node4
	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.doRequest(LBHttpSolrServer.java:343)
	at org.apache.solr.client.solrj.impl.LBHttpSolrServer.request(LBHttpSolrServer.java:304)
	... 47 more




Build Log:
[...truncated 10877 lines...]
   [junit4] Suite: org.apache.solr.cloud.DeleteReplicaTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\init-core-data-001
   [junit4]   2> 1226895 T2716 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (true)
   [junit4]   2> 1226895 T2716 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 1226899 T2716 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 1226900 T2716 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1226901 T2717 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 1226981 T2716 oasc.ZkTestServer.run start zk server on port:64233
   [junit4]   2> 1226981 T2716 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1226985 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1226991 T2723 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@83fdac name:ZooKeeperConnection Watcher:127.0.0.1:64233 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1226992 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1226992 T2716 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1226992 T2716 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 1226999 T2716 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1227001 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1227003 T2725 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69416a name:ZooKeeperConnection Watcher:127.0.0.1:64233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1227004 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1227004 T2716 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1227004 T2716 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 1227011 T2716 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 1227015 T2716 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 1227021 T2716 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 1227026 T2716 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1227026 T2716 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1227032 T2716 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1227032 T2716 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 1227038 T2716 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1227038 T2716 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1227043 T2716 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1227043 T2716 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1227048 T2716 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1227048 T2716 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1227054 T2716 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1227054 T2716 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 1227059 T2716 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1227059 T2716 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1227064 T2716 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1227064 T2716 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1227068 T2716 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1227068 T2716 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1227074 T2716 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1227074 T2716 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1227080 T2716 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1227081 T2716 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1228507 T2716 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1228515 T2716 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:64240
   [junit4]   2> 1228515 T2716 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1228515 T2716 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1228515 T2716 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-002
   [junit4]   2> 1228515 T2716 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-002\'
   [junit4]   2> 1228545 T2716 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-002\solr.xml
   [junit4]   2> 1228585 T2716 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-002
   [junit4]   2> 1228585 T2716 oasc.CoreContainer.<init> New CoreContainer 33362955
   [junit4]   2> 1228585 T2716 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-002\]
   [junit4]   2> 1228587 T2716 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1228587 T2716 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1228587 T2716 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1228587 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1228587 T2716 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1228587 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1228587 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1228587 T2716 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1228587 T2716 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1228590 T2716 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1228590 T2716 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1228590 T2716 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1228590 T2716 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1228590 T2716 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:64233/solr
   [junit4]   2> 1228590 T2716 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1228590 T2716 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1228593 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1228594 T2736 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2f1383 name:ZooKeeperConnection Watcher:127.0.0.1:64233 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1228595 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1228595 T2716 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1228599 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1228600 T2738 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fdabab name:ZooKeeperConnection Watcher:127.0.0.1:64233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1228600 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1228602 T2716 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 1228607 T2716 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 1228611 T2716 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 1228614 T2716 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 1228619 T2716 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 1228624 T2716 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 1228627 T2716 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64240_
   [junit4]   2> 1228628 T2716 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:64240_
   [junit4]   2> 1228633 T2716 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 1228636 T2716 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 1228639 T2716 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1228642 T2716 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:64240_
   [junit4]   2> 1228642 T2716 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 1228645 T2716 oasc.Overseer.start Overseer (id=92456901443452931-127.0.0.1:64240_-n_0000000000) starting
   [junit4]   2> 1228649 T2716 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 1228663 T2716 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1228665 T2740 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 1228666 T2716 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 1228669 T2716 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 1228672 T2716 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1228679 T2739 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 1228681 T2716 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-002
   [junit4]   2> 1229263 T2716 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-002\conf\
   [junit4]   2> 1229364 T2716 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1229366 T2716 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 1229366 T2716 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1229368 T2716 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1229370 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1229372 T2744 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2d1035 name:ZooKeeperConnection Watcher:127.0.0.1:64233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1229372 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1229373 T2716 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1229375 T2716 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1229382 T2716 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1229382 T2716 oasc.AbstractFullDistribZkTestBase.createJettys Creating collection1 with stateFormat=2
   [junit4]   2> 1229382 T2716 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1229384 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1229385 T2746 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8635b1 name:ZooKeeperConnection Watcher:127.0.0.1:64233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1229386 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1229386 T2716 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1229391 T2747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1229396 T2739 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: collection1
   [junit4]   2> 1229396 T2739 oasc.Overseer$ClusterStateUpdater.createCollection Create collection collection1 with shards [shard1, shard2]
   [junit4]   2> 1229396 T2739 oasc.Overseer$ClusterStateUpdater.createCollection state version collection1 2
   [junit4]   2> 1229402 T2739 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/collection1/state.json
   [junit4]   2> 1229406 T2747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1229412 T2748 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 1230791 T2716 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 1230791 T2716 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1230795 T2716 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:64255
   [junit4]   2> 1230797 T2716 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1230797 T2716 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1230797 T2716 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-003
   [junit4]   2> 1230798 T2716 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-003\'
   [junit4]   2> 1230826 T2716 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-003\solr.xml
   [junit4]   2> 1230881 T2716 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-003
   [junit4]   2> 1230881 T2716 oasc.CoreContainer.<init> New CoreContainer 22393952
   [junit4]   2> 1230882 T2716 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-003\]
   [junit4]   2> 1230883 T2716 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1230883 T2716 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1230884 T2716 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1230884 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1230884 T2716 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1230884 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1230884 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1230885 T2716 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1230885 T2716 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1230886 T2716 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1230886 T2716 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1230886 T2716 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1230887 T2716 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1230887 T2716 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:64233/solr
   [junit4]   2> 1230887 T2716 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1230887 T2716 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1230890 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1230892 T2759 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b520da name:ZooKeeperConnection Watcher:127.0.0.1:64233 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1230893 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1230894 T2716 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1230900 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1230902 T2761 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@bda455 name:ZooKeeperConnection Watcher:127.0.0.1:64233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1230902 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1230911 T2716 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1231713 T2716 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64255_
   [junit4]   2> 1231714 T2716 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:64255_
   [junit4]   2> 1231722 T2716 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1231728 T2716 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-003
   [junit4]   2> 1232714 T2716 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-003\conf\
   [junit4]   2> 1232897 T2716 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1232899 T2716 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 1232899 T2716 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1234253 T2716 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 1234254 T2716 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1234259 T2716 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:64264
   [junit4]   2> 1234259 T2716 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1234261 T2716 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1234261 T2716 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-004
   [junit4]   2> 1234261 T2716 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-004\'
   [junit4]   2> 1234291 T2716 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-004\solr.xml
   [junit4]   2> 1234331 T2716 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-004
   [junit4]   2> 1234331 T2716 oasc.CoreContainer.<init> New CoreContainer 13801278
   [junit4]   2> 1234332 T2716 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-004\]
   [junit4]   2> 1234334 T2716 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1234334 T2716 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1234334 T2716 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1234334 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1234334 T2716 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1234334 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1234334 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1234334 T2716 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1234335 T2716 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1234335 T2716 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1234335 T2716 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1234335 T2716 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1234335 T2716 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1234337 T2716 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:64233/solr
   [junit4]   2> 1234337 T2716 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1234337 T2716 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1234339 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1234340 T2774 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c78390 name:ZooKeeperConnection Watcher:127.0.0.1:64233 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1234341 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1234342 T2716 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1234344 T2718 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x148790fdf920008, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 1234345 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1234346 T2776 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@130e843 name:ZooKeeperConnection Watcher:127.0.0.1:64233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1234347 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1234355 T2716 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1235227 T2716 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64264_
   [junit4]   2> 1235230 T2716 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:64264_
   [junit4]   2> 1235242 T2716 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1235249 T2716 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-004
   [junit4]   2> 1235904 T2716 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-004\conf\
   [junit4]   2> 1236010 T2716 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1236012 T2716 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 1236012 T2716 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1237639 T2716 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 1237640 T2716 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1237649 T2716 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:64273
   [junit4]   2> 1237650 T2716 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1237650 T2716 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1237650 T2716 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005
   [junit4]   2> 1237650 T2716 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005\'
   [junit4]   2> 1237685 T2716 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005\solr.xml
   [junit4]   2> 1237732 T2716 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005
   [junit4]   2> 1237733 T2716 oasc.CoreContainer.<init> New CoreContainer 12121304
   [junit4]   2> 1237733 T2716 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005\]
   [junit4]   2> 1237734 T2716 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1237734 T2716 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1237735 T2716 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1237735 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1237735 T2716 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1237736 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1237736 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1237737 T2716 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1237738 T2716 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1237738 T2716 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1237738 T2716 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1237738 T2716 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1237739 T2716 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1237739 T2716 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:64233/solr
   [junit4]   2> 1237739 T2716 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1237740 T2716 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1237744 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1237749 T2789 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@d2b948 name:ZooKeeperConnection Watcher:127.0.0.1:64233 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1237750 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1237750 T2716 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1237756 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1237758 T2791 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1f2539e name:ZooKeeperConnection Watcher:127.0.0.1:64233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1237758 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1237773 T2716 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1238656 T2716 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64273_
   [junit4]   2> 1238658 T2716 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:64273_
   [junit4]   2> 1238672 T2716 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1238679 T2716 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005
   [junit4]   2> 1239291 T2716 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005\conf\
   [junit4]   2> 1239414 T2716 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1239415 T2716 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 1239415 T2716 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1240998 T2716 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 1240998 T2716 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1241004 T2716 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:64282
   [junit4]   2> 1241005 T2716 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1241005 T2716 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1241005 T2716 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-006
   [junit4]   2> 1241006 T2716 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-006\'
   [junit4]   2> 1241035 T2716 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-006\solr.xml
   [junit4]   2> 1241108 T2716 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-006
   [junit4]   2> 1241108 T2716 oasc.CoreContainer.<init> New CoreContainer 4794936
   [junit4]   2> 1241108 T2716 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-006\]
   [junit4]   2> 1241112 T2716 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1241112 T2716 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 1241112 T2716 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1241112 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1241112 T2716 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1241112 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1241112 T2716 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1241114 T2716 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1241114 T2716 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1241114 T2716 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 1241116 T2716 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1241116 T2716 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1241116 T2716 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1241118 T2716 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:64233/solr
   [junit4]   2> 1241118 T2716 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 1241118 T2716 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1241121 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1241123 T2804 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@172eaaf name:ZooKeeperConnection Watcher:127.0.0.1:64233 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1241124 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1241125 T2716 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1241128 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1241130 T2806 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1582550 name:ZooKeeperConnection Watcher:127.0.0.1:64233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1241131 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1241143 T2716 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1242026 T2716 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64282_
   [junit4]   2> 1242028 T2716 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:64282_
   [junit4]   2> 1242042 T2716 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 1242051 T2716 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-006
   [junit4]   2> 1243114 T2716 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-006\conf\
   [junit4]   2> 1243304 T2716 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 1243307 T2716 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 1243307 T2716 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1243312 T2716 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1243313 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1243314 T2810 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1564a75 name:ZooKeeperConnection Watcher:127.0.0.1:64233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1243315 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1243315 T2716 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1243318 T2716 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1243325 T2716 oasc.AbstractFullDistribZkTestBase.createCollection Creating collection with stateFormat=2: delLiveColl
   [junit4]   2> 1243325 T2716 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 1243327 T2716 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 1243328 T2812 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b88624 name:ZooKeeperConnection Watcher:127.0.0.1:64233/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1243328 T2716 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 1243329 T2716 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 1243331 T2716 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 1243344 T2733 oasha.CollectionsHandler.handleCreateAction Creating Collection : stateFormat=2&action=CREATE&version=2&maxShardsPerNode=2&wt=javabin&name=delLiveColl&replicationFactor=2&numShards=2
   [junit4]   2> 1243347 T2747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/collection-queue-work state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1243349 T2740 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delLiveColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2",
   [junit4]   2> 	  "stateFormat":"2"}
   [junit4]   2> 1243351 T2813 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : createcollection , {
   [junit4]   2> 	  "operation":"createcollection",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delLiveColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2",
   [junit4]   2> 	  "stateFormat":"2"}
   [junit4]   2> 1243355 T2813 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
   [junit4]   2> 1243355 T2813 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/delLiveColl 
   [junit4]   2> 1243356 T2813 oascc.SolrZkClient.makePath makePath: /collections/delLiveColl
   [junit4]   2> 1243364 T2747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1243372 T2739 oasc.Overseer$ClusterStateUpdater.buildCollection building a new collection: delLiveColl
   [junit4]   2> 1243373 T2739 oasc.Overseer$ClusterStateUpdater.createCollection Create collection delLiveColl with shards [shard1, shard2]
   [junit4]   2> 1243373 T2739 oasc.Overseer$ClusterStateUpdater.createCollection state version delLiveColl 2
   [junit4]   2> 1243377 T2739 oasc.Overseer$ClusterStateUpdater.updateZkStates going to create_collection /collections/delLiveColl/state.json
   [junit4]   2> 1243380 T2748 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1243381 T2747 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1243382 T2777 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1243383 T2807 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1243383 T2762 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1243383 T2792 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1243384 T2815 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1243384 T2814 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 1243461 T2813 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection delLiveColl, shardNames [shard1, shard2] , replicationFactor : 2
   [junit4]   2> 1243461 T2813 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica1 as part of slice shard1 of collection delLiveColl on 127.0.0.1:64282_
   [junit4]   2> 1243463 T2813 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard1_replica2 as part of slice shard1 of collection delLiveColl on 127.0.0.1:64273_
   [junit4]   2> 1243465 T2813 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica1 as part of slice shard2 of collection delLiveColl on 127.0.0.1:64240_
   [junit4]   2> 1243468 T2800 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&action=CREATE&version=2&shard=shard1&collection.configName=conf1&wt=javabin&name=delLiveColl_shard1_replica1&collection=delLiveColl&numShards=2
   [junit4]   2> 1243468 T2784 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&action=CREATE&version=2&shard=shard1&collection.configName=conf1&wt=javabin&name=delLiveColl_shard1_replica2&collection=delLiveColl&numShards=2
   [junit4]   2> 1243471 T2813 oasc.OverseerCollectionProcessor.createCollection Creating shard delLiveColl_shard2_replica2 as part of slice shard2 of collection delLiveColl on 127.0.0.1:64264_
   [junit4]   2> 1243475 T2768 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&action=CREATE&version=2&shard=shard2&collection.configName=conf1&wt=javabin&name=delLiveColl_shard2_replica2&collection=delLiveColl&numShards=2
   [junit4]   2> 1243481 T2734 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&action=CREATE&version=2&shard=shard2&collection.configName=conf1&wt=javabin&name=delLiveColl_shard2_replica1&collection=delLiveColl&numShards=2
   [junit4]   2> 1243482 T2784 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica2 state=down collection=delLiveColl
   [junit4]   2> 1243483 T2800 oasc.ZkController.publish publishing core=delLiveColl_shard1_replica1 state=down collection=delLiveColl
   [junit4]   2> 1243485 T2768 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica2 state=down collection=delLiveColl
   [junit4]   2> 1243487 T2747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1243488 T2734 oasc.ZkController.publish publishing core=delLiveColl_shard2_replica1 state=down collection=delLiveColl
   [junit4]   2> 1243494 T2784 oasc.ZkController.preRegister Registering watch for external collection delLiveColl
   [junit4]   2> 1243494 T2784 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl
   [junit4]   2> 1243494 T2800 oasc.ZkController.preRegister Registering watch for external collection delLiveColl
   [junit4]   2> 1243494 T2800 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl
   [junit4]   2> 1243496 T2784 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... 
   [junit4]   2> 1243497 T2768 oasc.ZkController.preRegister Registering watch for external collection delLiveColl
   [junit4]   2> 1243497 T2768 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl
   [junit4]   2> 1243498 T2800 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... 
   [junit4]   2> 1243500 T2768 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... 
   [junit4]   2> 1243504 T2784 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 
   [junit4]   2> 1243504 T2784 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1243505 T2734 oasc.ZkController.preRegister Registering watch for external collection delLiveColl
   [junit4]   2> 1243507 T2739 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:64273",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "node_name":"127.0.0.1:64273_",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "core":"delLiveColl_shard1_replica2",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "numShards":"2"}
   [junit4]   2> 1243507 T2800 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 
   [junit4]   2> 1243508 T2800 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1243509 T2768 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 0 
   [junit4]   2> 1243509 T2768 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1243526 T2739 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json
   [junit4]   2> 1243527 T2807 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243529 T2777 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243529 T2792 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243530 T2807 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 1243535 T2777 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 1243535 T2792 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 1243537 T2734 oascc.ZkStateReader.addZkWatch addZkWatch delLiveColl
   [junit4]   2> 1243538 T2734 oascc.ZkStateReader.addZkWatch Updating collection state at /collections/delLiveColl/state.json from ZooKeeper... 
   [junit4]   2> 1243539 T2734 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 1243539 T2734 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 1243550 T2739 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 1 
   [junit4]   2> 1243551 T2739 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:64282",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "node_name":"127.0.0.1:64282_",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "core":"delLiveColl_shard1_replica1",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "numShards":"2"}
   [junit4]   2> 1243554 T2747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1243555 T2739 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json
   [junit4]   2> 1243556 T2807 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243556 T2792 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243558 T2777 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243558 T2747 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243559 T2792 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 1243559 T2807 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 1243560 T2777 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 1243563 T2747 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 1243571 T2739 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 2 
   [junit4]   2> 1243572 T2739 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:64264",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "node_name":"127.0.0.1:64264_",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "core":"delLiveColl_shard2_replica2",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "numShards":"2"}
   [junit4]   2> 1243575 T2747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1243576 T2739 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json
   [junit4]   2> 1243578 T2792 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243578 T2807 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243579 T2777 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243580 T2792 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 
   [junit4]   2> 1243579 T2747 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243580 T2807 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 
   [junit4]   2> 1243581 T2777 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 
   [junit4]   2> 1243584 T2747 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 
   [junit4]   2> 1243591 T2739 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 3 
   [junit4]   2> 1243591 T2739 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=2 message={
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:64240",
   [junit4]   2> 	  "collection":"delLiveColl",
   [junit4]   2> 	  "node_name":"127.0.0.1:64240_",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "core":"delLiveColl_shard2_replica1",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "numShards":"2"}
   [junit4]   2> 1243595 T2747 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 1243595 T2739 oasc.Overseer$ClusterStateUpdater.updateZkStates going to update_collection /collections/delLiveColl/state.json
   [junit4]   2> 1243597 T2792 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243597 T2777 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243598 T2807 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243599 T2747 oascc.ZkStateReader$7.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delLiveColl/state.json, has occurred - updating... 
   [junit4]   2> 1243599 T2792 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 
   [junit4]   2> 1243599 T2777 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 
   [junit4]   2> 1243600 T2807 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 
   [junit4]   2> 1243603 T2747 oascc.ZkStateReader.updateWatchedCollection Updating data for delLiveColl to ver 4 
   [junit4]   2> 1244465 T2784 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica2
   [junit4]   2> 1244465 T2784 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
   [junit4]   2> 1244467 T2784 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1244467 T2784 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 1244469 T2800 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard1_replica1
   [junit4]   2> 1244469 T2800 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
   [junit4]   2> 1244470 T2784 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1244470 T2784 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005\delLiveColl_shard1_replica2\'
   [junit4]   2> 1244470 T2768 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica2
   [junit4]   2> 1244471 T2768 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
   [junit4]   2> 1244471 T2800 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1244472 T2800 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 1244474 T2768 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1244474 T2768 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 1244476 T2800 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1244476 T2800 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-006\delLiveColl_shard1_replica1\'
   [junit4]   2> 1244479 T2768 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1244479 T2768 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-004\delLiveColl_shard2_replica2\'
   [junit4]   2> 1244503 T2734 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delLiveColl_shard2_replica1
   [junit4]   2> 1244503 T2734 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delLiveColl
   [junit4]   2> 1244527 T2734 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 1244527 T2734 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delLiveColl
   [junit4]   2> 1244534 T2734 oascc.ZkStateReader.readConfigName path=/collections/delLiveColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 1244534 T2734 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-002\delLiveColl_shard2_replica1\'
   [junit4]   2> 1244596 T2784 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1244597 T2800 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1244642 T2768 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1244669 T2800 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1244701 T2768 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1244703 T2800 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1244711 T2784 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1244715 T2800 oass.IndexSchema.readSchema [delLiveColl_shard1_replica1] Schema name=test
   [junit4]   2> 1244719 T2768 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1244729 T2768 oass.IndexSchema.readSchema [delLiveColl_shard2_replica2] Schema name=test
   [junit4]   2> 1244745 T2734 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 1244804 T2734 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1244805 T2784 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1244815 T2784 oass.IndexSchema.readSchema [delLiveColl_shard1_replica2] Schema name=test
   [junit4]   2> 1245175 T2734 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 1245187 T2734 oass.IndexSchema.readSchema [delLiveColl_shard2_replica1] Schema name=test
   [junit4]   2> 1245934 T2784 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1245939 T2800 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1245949 T2784 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1245954 T2800 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1245958 T2784 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1245964 T2800 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1246072 T2784 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1246083 T2784 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 1246105 T2768 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1246112 T2768 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 1246115 T2768 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1246124 T2784 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1246129 T2784 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1246129 T2784 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1246129 T2784 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1246136 T2784 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 1246136 T2784 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 1246137 T2784 oasc.CoreContainer.create Creating SolrCore 'delLiveColl_shard1_replica2' using configuration from collection delLiveColl
   [junit4]   2> 1246137 T2784 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1246138 T2784 oasc.SolrCore.<init> [delLiveColl_shard1_replica2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005\delLiveColl_shard1_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005\delLiveColl_shard1_replica2\data\
   [junit4]   2> 1246138 T2784 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@10163f6
   [junit4]   2> 1246140 T2784 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005\delLiveColl_shard1_replica2\data\
   [junit4]   2> 1246141 T2784 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005\delLiveColl_shard1_replica2\data\index/
   [junit4]   2> 1246141 T2784 oasc.SolrCore.initIndex WARN [delLiveColl_shard1_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005\delLiveColl_shard1_replica2\data\index' doesn't exist. Creating new index...
   [junit4]   2> 1246142 T2784 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteReplicaTest-40015FF1CB47EE66-001\tempDir-005\delLiveColl_shard1_replica2\data\index
   [junit4]   2> 1246143 T2784 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=17, maxMergeAtOnceExplicit=30, maxMergedSegmentMB=14.45703125, floorSegmentMB=1.69140625, forceMergeDeletesPctAllowed=8.899551040512542, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3486525566445853
   [junit4]   2> 1246143 T2784 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@14d9ffa lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7eaddf),segFN=segments_1,generation=1}
   [junit4]   2> 1246143 T2784 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1246149 T2784 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1246149 T2784 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1246150 T2784 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1246150 T2784 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1246150 T2784 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1246151 T2784 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1246152 T2784 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1246152 T2784 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1246152 T2784 oasc.SolrCore.loadUpdateProcessorChains no upd

[...truncated too long message...]

unit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader.access$200(ZkStateReader.java:55)
   [junit4]   2> 	at org.apache.solr.common.cloud.ZkStateReader$2.process(ZkStateReader.java:315)
   [junit4]   2> 	at org.apache.solr.common.cloud.SolrZkClient$3$1.run(SolrZkClient.java:266)
   [junit4]   2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   [junit4]   2> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]   2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 4338042 T8472 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 4338062 T7724 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/be,null}
   [junit4]   2> 4344312 T7724 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 4344313 T7724 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:55825 55825
   [junit4]   2> 4347137 T7725 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:55825 55825
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.method=testDistribSearch -Dtests.seed=40015FF1CB47EE66 -Dtests.slow=true -Dtests.locale=zh -Dtests.timezone=Africa/Porto-Novo -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR    803s | CollectionsAPIDistributedZkTest.testDistribSearch <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: Could not get shard id for core: halfcollection_shard1_replica1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([40015FF1CB47EE66:C1E7D1E9BC188E5A]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.executeMethod(HttpSolrServer.java:550)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:210)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:206)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:583)
   [junit4]    > 	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.doTest(CollectionsAPIDistributedZkTest.java:205)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:869)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 4347162 T7724 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CollectionsAPIDistributedZkTest-40015FF1CB47EE66-001
   [junit4]   2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=108), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=108)), sim=DefaultSimilarity, locale=zh, timezone=Africa/Porto-Novo
   [junit4]   2> NOTE: Windows 7 6.1 x86/Oracle Corporation 1.7.0_67 (32-bit)/cpus=2,threads=1,free=117008136,total=264413184
   [junit4]   2> NOTE: All tests run in this JVM: [ShardRoutingCustomTest, TestBinaryResponseWriter, QueryElevationComponentTest, TestManagedStopFilterFactory, LukeRequestHandlerTest, NotRequiredUniqueKeyTest, BJQParserTest, SuggesterFSTTest, AssignTest, HdfsChaosMonkeySafeLeaderTest, LoggingHandlerTest, TestIndexSearcher, TestComponentsName, TestBM25SimilarityFactory, TestSolrXml, TestRangeQuery, HdfsCollectionsAPIDistributedZkTest, TestSolrXmlPersistor, SolrCoreCheckLockOnStartupTest, TestCodecSupport, SchemaVersionSpecificBehaviorTest, RequestHandlersTest, BadIndexSchemaTest, SimpleFacetsTest, QueryParsingTest, TestReplicationHandler, TestAnalyzeInfixSuggestions, TestNRTOpen, WordBreakSolrSpellCheckerTest, TestPivotHelperCode, TestRecoveryHdfs, TestLazyCores, DisMaxRequestHandlerTest, TestCSVLoader, SignatureUpdateProcessorFactoryTest, UniqFieldsUpdateProcessorFactoryTest, TestDocSet, SOLR749Test, StatelessScriptUpdateProcessorFactoryTest, TestCloudPivotFacet, TestJoin, DistributedIntervalFacetingTest, TestHighFrequencyDictionaryFactory, CurrencyFieldXmlFileTest, TestSurroundQueryParser, MergeStrategyTest, CoreAdminCreateDiscoverTest, FastVectorHighlighterTest, TestRandomDVFaceting, QueryEqualityTest, TestUtils, TestFunctionQuery, XsltUpdateRequestHandlerTest, TestFoldingMultitermQuery, TestFastOutputStream, TestFiltering, HttpPartitionTest, ZkControllerTest, TestManagedSchemaDynamicFieldResource, ReplicationFactorTest, RollingRestartTest, TestGroupingSearch, CollectionsAPIAsyncDistributedZkTest, TestCollationField, TestShortCircuitedRequests, AnalyticsMergeStrategyTest, TestWriterPerf, TestFastWriter, CursorMarkTest, DebugComponentTest, TestCopyFieldCollectionResource, RangeFacetTest, TestMissingGroups, TimeZoneUtilsTest, DistributedSpellCheckComponentTest, SystemInfoHandlerTest, TestFieldCollectionResource, TestUpdate, AliasIntegrationTest, AbstractAnalyticsFacetTest, ConnectionManagerTest, TestConfig, TestQueryTypes, TestMergePolicyConfig, UUIDFieldTest, DeleteReplicaTest, HdfsBasicDistributedZk2Test, TestLRUCache, RegexBoostProcessorTest, TestDynamicFieldCollectionResource, TestInitQParser, TestStressReorder, ResponseHeaderTest, TestDocBasedVersionConstraints, TestJmxMonitoredMap, TestMaxScoreQueryParser, QueryResultKeyTest, BasicFunctionalityTest, CacheHeaderTest, SSLMigrationTest, HdfsWriteToMultipleCollectionsTest, TriLevelCompositeIdRoutingTest, MigrateRouteKeyTest, TestQueryUtils, MultiTermTest, TestDistributedSearch, TestCoreDiscovery, HdfsUnloadDistributedZkTest, ParsingFieldUpdateProcessorsTest, InfoHandlerTest, TestTolerantSearch, AnalysisErrorHandlingTest, TestBadConfig, IndexSchemaTest, PluginInfoTest, DocExpirationUpdateProcessorFactoryTest, TestReRankQParserPlugin, RankQueryTest, TestClassNameShortening, DateFieldTest, CustomCollectionTest, RecoveryZkTest, TestFaceting, TestPostingsSolrHighlighter, TestAddFieldRealTimeGet, PathHierarchyTokenizerFactoryTest, CoreMergeIndexesAdminHandlerTest, TestExceedMaxTermLength, PolyFieldTest, HighlighterTest, TestArbitraryIndexDir, DirectSolrConnectionTest, AbstractAnalyticsStatsTest, DocValuesMissingTest, SliceStateTest, SharedFSAutoReplicaFailoverUtilsTest, TestSimpleQParserPlugin, TestCloudSchemaless, TestCloudInspectUtil, TestFieldSortValues, TestRandomFaceting, TestRandomMergePolicy, SolrCoreTest, ChaosMonkeySafeLeaderTest, TestNoOpRegenerator, UnloadDistributedZkTest, BlockCacheTest, SynonymTokenizerTest, SolrPluginUtilsTest, HdfsDirectoryTest, SyncSliceTest, SearchHandlerTest, TestHighlightDedupGrouping, ResourceLoaderTest, TestManagedSynonymFilterFactory, TestDefaultSimilarityFactory, TestAnalyzedSuggestions, TestQuerySenderListener, TestReplicationHandlerBackup, BasicDistributedZkTest, TestFreeTextSuggestions, TestAtomicUpdateErrorCases, TestSolrXmlPersistence, TestParamSet, PrimUtilsTest, ModifyConfFileTest, SolrIndexSplitterTest, DOMUtilTest, DateMathParserTest, SuggesterTest, CircularListTest, LegacyHTMLStripCharFilterTest, SuggestComponentTest, TestMiniSolrCloudCluster, HighlighterConfigTest, TestStandardQParsers, SolrInfoMBeanTest, TestSchemaResource, TestValueSourceCache, TestSolrJ, TestChildDocTransformer, PreAnalyzedFieldTest, TestPseudoReturnFields, OutOfBoxZkACLAndCredentialsProvidersTest, TestManagedSchemaFieldResource, TestSolr4Spatial, DistanceFunctionTest, TestManagedResourceStorage, SpellingQueryConverterTest, SpellCheckCollatorTest, SpellPossibilityIteratorTest, PeerSyncTest, TestInfoStreamLogging, HdfsBasicDistributedZkTest, TestFileDictionaryLookup, LeaderElectionTest, SharedFSAutoReplicaFailoverTest, SoftAutoCommitTest, AtomicUpdatesTest, ZkCLITest, TestShardHandlerFactory, TestLuceneMatchVersion, TestDistributedMissingSort, ScriptEngineTest, DeleteLastCustomShardedReplicaTest, TestCollapseQParserPlugin, TestManagedSchemaFieldTypeResource, SpellCheckComponentTest, DeleteShardTest, OverseerTest, DistributedDebugComponentTest, HdfsLockFactoryTest, BasicDistributedZk2Test, CollectionsAPIDistributedZkTest]
   [junit4] Completed in 803.55s, 1 test, 1 error <<< FAILURES!

[...truncated 699 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:491: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:471: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:60: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:496: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1358: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:961: There were test failures: 438 suites, 1830 tests, 3 errors, 54 ignored (32 assumptions)

Total time: 181 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.7.0_67 -client -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.7.0_67) - Build # 4312 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4312/
Java: 32bit/jdk1.7.0_67 -client -XX:+UseConcMarkSweepGC

5 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestZkChroot

Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=6460, name=coreLoadExecutor-3852-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below.

Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=6460, name=coreLoadExecutor-3852-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below.
	at java.lang.Thread.getStackTrace(Thread.java:1589)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:166)
	at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:731)
	at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:1274)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1165)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1157)
	at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:40)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:276)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:488)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError: Directory not closed: MockDirectoryWrapper(RAMDirectory@12e9f4d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@13487fd)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:699)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:696)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:183)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:712)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	... 1 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestZkChroot

Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=6480, name=coreLoadExecutor-3859-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below.

Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=6480, name=coreLoadExecutor-3859-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below.
	at java.lang.Thread.getStackTrace(Thread.java:1589)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:166)
	at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:731)
	at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:1274)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1165)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1157)
	at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:40)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:276)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:488)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError: Directory not closed: MockDirectoryWrapper(RAMDirectory@1801d44 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@5cfd92)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:699)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:696)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:183)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:712)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	... 1 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.TestZkChroot

Error Message:
Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=6498, name=coreLoadExecutor-3866-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below.

Stack Trace:
com.carrotsearch.randomizedtesting.ResourceDisposalError: Resource in scope SUITE failed to close. Resource was registered from thread Thread[id=6498, name=coreLoadExecutor-3866-thread-1, state=RUNNABLE, group=TGRP-TestZkChroot], registration stack trace below.
	at java.lang.Thread.getStackTrace(Thread.java:1589)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeAtEnd(RandomizedContext.java:166)
	at org.apache.lucene.util.LuceneTestCase.closeAfterSuite(LuceneTestCase.java:731)
	at org.apache.lucene.util.LuceneTestCase.wrapDirectory(LuceneTestCase.java:1274)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1165)
	at org.apache.lucene.util.LuceneTestCase.newDirectory(LuceneTestCase.java:1157)
	at org.apache.solr.core.MockDirectoryFactory.create(MockDirectoryFactory.java:40)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:276)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:488)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.AssertionError: Directory not closed: MockDirectoryWrapper(RAMDirectory@274403 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@10ba15b)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:699)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2$1.apply(RandomizedRunner.java:696)
	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:183)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$2.afterAlways(RandomizedRunner.java:712)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	... 1 more


FAILED:  org.apache.solr.core.CachingDirectoryFactoryTest.stressTest

Error Message:
Captured an uncaught exception in thread: Thread[id=3913, name=Thread-1565, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3913, name=Thread-1565, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
	at java.security.AccessController.checkPermission(AccessController.java:559)
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
	at java.nio.file.Files.createDirectory(Files.java:630)
	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
	at java.nio.file.Files.createDirectories(Files.java:683)
	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)


FAILED:  org.apache.solr.core.RAMDirectoryFactoryTest.test

Error Message:
access denied ("java.io.FilePermission" "\fake\path" "write")

Stack Trace:
java.security.AccessControlException: access denied ("java.io.FilePermission" "\fake\path" "write")
	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD:5380019E9591BF05]:0)
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
	at java.security.AccessController.checkPermission(AccessController.java:559)
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
	at java.nio.file.Files.createDirectory(Files.java:630)
	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
	at java.nio.file.Files.createDirectories(Files.java:683)
	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
	at org.apache.solr.core.RAMDirectoryFactoryTest.dotestOpenReturnsTheSameForSamePath(RAMDirectoryFactoryTest.java:46)
	at org.apache.solr.core.RAMDirectoryFactoryTest.test(RAMDirectoryFactoryTest.java:33)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 10866 lines...]
   [junit4] Suite: org.apache.solr.core.CachingDirectoryFactoryTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.core.CachingDirectoryFactoryTest-DBD43E443B6DD2FD-001\init-core-data-001
   [junit4]   2> 1790292 T3912 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 1790296 T3912 oas.SolrTestCaseJ4.setUp ###Starting stressTest
   [junit4]   2> 1790662 T3913 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@1ba49d1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9053a assuming 'simple'
   [junit4]   2> 1790671 T3917 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@1de2296 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@f8a6d assuming 'simple'
   [junit4]   2> 1790672 T3919 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@4939bc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1d63ff6 assuming 'simple'
   [junit4]   2> 1790673 T3922 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@cda7b3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1dc5f83 assuming 'simple'
   [junit4]   2> 1790673 T3918 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@1029ba4 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@19fc18f assuming 'simple'
   [junit4]   2> 1790674 T3916 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@15cc247 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@eb7183 assuming 'simple'
   [junit4]   2> 1790676 T3914 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@16971c7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@10cd827 assuming 'simple'
   [junit4]   2> 1790677 T3920 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@3529b0 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@fd3d8 assuming 'simple'
   [junit4]   2> 385 T3917 ccr.RandomizedRunner$QueueUncaughtExceptionsHandler.uncaughtException WARNING Uncaught exception in thread: Thread[Thread-1569,5,TGRP-CachingDirectoryFactoryTest] java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]   2> 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]   2> 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]   2> 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]   2> 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)
   [junit4]   2> 
   [junit4]   2> 1790679 T3915 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@139ba97 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@f1fbc assuming 'simple'
   [junit4]   2> 385 T3922 ccr.RandomizedRunner$QueueUncaughtExceptionsHandler.uncaughtException WARNING Uncaught exception in thread: Thread[Thread-1574,5,TGRP-CachingDirectoryFactoryTest] java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]   2> 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]   2> 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]   2> 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]   2> 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)
   [junit4]   2> 
   [junit4]   2> 386 T3913 ccr.RandomizedRunner$QueueUncaughtExceptionsHandler.uncaughtException WARNING Uncaught exception in thread: Thread[Thread-1565,5,TGRP-CachingDirectoryFactoryTest] java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]   2> 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]   2> 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]   2> 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]   2> 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)
   [junit4]   2> 
   [junit4]   2> 386 T3916 ccr.RandomizedRunner$QueueUncaughtExceptionsHandler.uncaughtException WARNING Uncaught exception in thread: Thread[Thread-1568,5,TGRP-CachingDirectoryFactoryTest] java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]   2> 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]   2> 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]   2> 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]   2> 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)
   [junit4]   2> 
   [junit4]   2> 1790679 T3921 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@ecae3e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1c2b49c assuming 'simple'
   [junit4]   2> 388 T3921 ccr.RandomizedRunner$QueueUncaughtExceptionsHandler.uncaughtException WARNING Uncaught exception in thread: Thread[Thread-1573,5,TGRP-CachingDirectoryFactoryTest] java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]   2> 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]   2> 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]   2> 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]   2> 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)
   [junit4]   2> 
   [junit4]   2> 1790680 T3923 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@160f31c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@c710ea assuming 'simple'
   [junit4]   2> 386 T3919 ccr.RandomizedRunner$QueueUncaughtExceptionsHandler.uncaughtException WARNING Uncaught exception in thread: Thread[Thread-1571,5,TGRP-CachingDirectoryFactoryTest] java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]   2> 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]   2> 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]   2> 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]   2> 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)
   [junit4]   2> 
   [junit4]   2> 386 T3920 ccr.RandomizedRunner$QueueUncaughtExceptionsHandler.uncaughtException WARNING Uncaught exception in thread: Thread[Thread-1572,5,TGRP-CachingDirectoryFactoryTest] java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]   2> 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]   2> 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]   2> 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]   2> 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)
   [junit4]   2> 
   [junit4]   2> 390 T3923 ccr.RandomizedRunner$QueueUncaughtExceptionsHandler.uncaughtException WARNING Uncaught exception in thread: Thread[Thread-1575,5,TGRP-CachingDirectoryFactoryTest] java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]   2> 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]   2> 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]   2> 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]   2> 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)
   [junit4]   2> 
   [junit4]   2> 388 T3914 ccr.RandomizedRunner$QueueUncaughtExceptionsHandler.uncaughtException WARNING Uncaught exception in thread: Thread[Thread-1566,5,TGRP-CachingDirectoryFactoryTest] java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]   2> 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]   2> 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]   2> 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]   2> 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)
   [junit4]   2> 
   [junit4]   2> 387 T3918 ccr.RandomizedRunner$QueueUncaughtExceptionsHandler.uncaughtException WARNING Uncaught exception in thread: Thread[Thread-1570,5,TGRP-CachingDirectoryFactoryTest] java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]   2> 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]   2> 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]   2> 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]   2> 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)
   [junit4]   2> 
   [junit4]   2> 387 T3915 ccr.RandomizedRunner$QueueUncaughtExceptionsHandler.uncaughtException WARNING Uncaught exception in thread: Thread[Thread-1567,5,TGRP-CachingDirectoryFactoryTest] java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]   2> 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]   2> 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]   2> 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]   2> 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)
   [junit4]   2> 
   [junit4]   2> 1799517 T3930 oasc.CachingDirectoryFactory.close Closing RAMDirectoryFactory - 0 directories currently being tracked
   [junit4]   2> 1799603 T3912 oas.SolrTestCaseJ4.tearDown ###Ending stressTest
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CachingDirectoryFactoryTest -Dtests.method=stressTest -Dtests.seed=DBD43E443B6DD2FD -Dtests.slow=true -Dtests.locale=ar_SD -Dtests.timezone=America/Iqaluit -Dtests.file.encoding=Cp1252
   [junit4] ERROR   9.36s | CachingDirectoryFactoryTest.stressTest <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3913, name=Thread-1565, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
   [junit4]    > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]    > 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3914, name=Thread-1566, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
   [junit4]    > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]    > 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3916, name=Thread-1568, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
   [junit4]    > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]    > 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3917, name=Thread-1569, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
   [junit4]    > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]    > 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3918, name=Thread-1570, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
   [junit4]    > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]    > 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3922, name=Thread-1574, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
   [junit4]    > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]    > 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3920, name=Thread-1572, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
   [junit4]    > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]    > 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #8: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3919, name=Thread-1571, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
   [junit4]    > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]    > 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #9: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3915, name=Thread-1567, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
   [junit4]    > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]    > 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #10: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3921, name=Thread-1573, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
   [junit4]    > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]    > 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)Throwable #11: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=3923, name=Thread-1575, state=RUNNABLE, group=TGRP-CachingDirectoryFactoryTest]
   [junit4]    > Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "path1\8\12" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]    > 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactoryTest$GetDirThread.run(CachingDirectoryFactoryTest.java:190)
   [junit4]   2> 1799653 T3912 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=ar_SD, timezone=America/Iqaluit
   [junit4]   2> NOTE: Windows 7 6.1 x86/Oracle Corporation 1.7.0_67 (32-bit)/cpus=2,threads=1,free=185532568,total=373710848
   [junit4]   2> NOTE: All tests run in this JVM: [TestReplicationHandler, TestFreeTextSuggestions, SolrInfoMBeanTest, MinimalSchemaTest, URLClassifyProcessorTest, FunctionTest, TestArbitraryIndexDir, FieldFacetExtrasTest, TestSystemIdResolver, TestCopyFieldCollectionResource, DocumentAnalysisRequestHandlerTest, CurrencyFieldXmlFileTest, QueryParsingTest, TestRawResponseWriter, TestDynamicFieldCollectionResource, TestFastOutputStream, TestSolrXmlPersistence, IndexSchemaTest, BJQParserTest, UUIDFieldTest, TestDynamicFieldResource, TestCSVLoader, TestDistributedSearch, IndexBasedSpellCheckerTest, TestCloudManagedSchema, SolrTestCaseJ4Test, TestStressLucene, TestAnalyzedSuggestions, TestSolrDeletionPolicy1, SuggesterTSTTest, HdfsSyncSliceTest, TestSearcherReuse, NumericFieldsTest, TestOmitPositions, TestSolrXmlPersistor, TestLFUCache, TestDefaultSimilarityFactory, QueryFacetTest, TestFieldTypeResource, ShardSplitTest, RequestHandlersTest, TestManagedResourceStorage, TestFunctionQuery, AnalyticsQueryTest, DateFieldTest, UpdateRequestProcessorFactoryTest, SolrPluginUtilsTest, SpellPossibilityIteratorTest, MigrateRouteKeyTest, BadComponentTest, JsonLoaderTest, MoreLikeThisHandlerTest, TestFastWriter, ClusterStateTest, AnalysisErrorHandlingTest, CursorPagingTest, TriLevelCompositeIdRoutingTest, ShardRoutingTest, ZkCLITest, SharedFSAutoReplicaFailoverUtilsTest, TestQuerySenderListener, TestCSVResponseWriter, TestNonNRTOpen, TestMultiCoreConfBootstrap, TestQueryTypes, StatelessScriptUpdateProcessorFactoryTest, BasicDistributedZkTest, TestStressVersions, TestTrie, TestFiltering, TestUpdate, TestModifyConfFiles, LeaderElectionIntegrationTest, ConvertedLegacyTest, SpellingQueryConverterTest, TestDocumentBuilder, AssignTest, TestJmxIntegration, TestStressUserVersions, TestFaceting, MBeansHandlerTest, TestRecoveryHdfs, TestPivotHelperCode, StandardRequestHandlerTest, OpenExchangeRatesOrgProviderTest, DisMaxRequestHandlerTest, TestCoreContainer, TestExtendedDismaxParser, TestValueSourceCache, TestJoin, EchoParamsTest, SortByFunctionTest, DocValuesMultiTest, TestShortCircuitedRequests, MergeStrategyTest, TestCoreDiscovery, TestLMDirichletSimilarityFactory, ExternalCollectionsTest, HardAutoCommitTest, TestBlendedInfixSuggestions, DistributedQueryElevationComponentTest, ScriptEngineTest, TestMaxScoreQueryParser, CachingDirectoryFactoryTest]
   [junit4] Completed in 9.43s, 1 test, 1 error <<< FAILURES!

[...truncated 260 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestZkChroot
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestZkChroot-DBD43E443B6DD2FD-001\init-core-data-001
   [junit4]   2> 3106275 T6438 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 3106284 T6438 oas.SolrTestCaseJ4.setUp ###Starting testNoBootstrapConf
   [junit4]   2> 3106286 T6438 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3106287 T6439 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 3106374 T6438 oasc.ZkTestServer.run start zk server on port:52273
   [junit4]   2> 3106375 T6438 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3106376 T6438 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3106380 T6445 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@64beb0 name:ZooKeeperConnection Watcher:127.0.0.1:52273 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3106381 T6438 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3106381 T6438 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3106382 T6438 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\'
   [junit4]   2> 3106413 T6438 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\solr.xml
   [junit4]   2> 3106446 T6438 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr
   [junit4]   2> 3106447 T6438 oasc.CoreContainer.<init> New CoreContainer 6371933
   [junit4]   2> 3106447 T6438 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\]
   [junit4]   2> 3106449 T6438 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 3106449 T6438 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 3106449 T6438 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 3106450 T6438 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 3106450 T6438 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 3106450 T6438 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 3106451 T6438 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 3106451 T6438 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 3106451 T6438 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 3106451 T6438 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=0&connTimeout=0&retry=false
   [junit4]   2> 3106451 T6438 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3106452 T6438 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3106452 T6438 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 3106453 T6438 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52273/foo/bar2
   [junit4]   2> 3106453 T6438 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 3106453 T6438 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3106455 T6438 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3106457 T6447 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13ed3c4 name:ZooKeeperConnection Watcher:127.0.0.1:52273 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3106458 T6438 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3106458 T6438 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3106462 T6440 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x148766a75440001, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 3106465 T6438 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:52273 52273
   [junit4]   2> 3108288 T6439 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:52273 52273
   [junit4]   2> 3108289 T6438 oas.SolrTestCaseJ4.tearDown ###Ending testNoBootstrapConf
   [junit4]   2> 3108294 T6438 oas.SolrTestCaseJ4.setUp ###Starting testChrootBootstrap
   [junit4]   2> 3108296 T6438 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3108297 T6448 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 3108375 T6438 oasc.ZkTestServer.run start zk server on port:52282
   [junit4]   2> 3108375 T6438 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\'
   [junit4]   2> 3108394 T6438 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\solr.xml
   [junit4]   2> 3108426 T6438 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr
   [junit4]   2> 3108426 T6438 oasc.CoreContainer.<init> New CoreContainer 13594553
   [junit4]   2> 3108426 T6438 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\]
   [junit4]   2> 3108428 T6438 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 0
   [junit4]   2> 3108428 T6438 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: null
   [junit4]   2> 3108428 T6438 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 0
   [junit4]   2> 3108428 T6438 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 3108428 T6438 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 3108428 T6438 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 3108428 T6438 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 3108428 T6438 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 3108428 T6438 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 3108428 T6438 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=0&connTimeout=0&retry=false
   [junit4]   2> 3108430 T6438 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 3108430 T6438 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 3108430 T6438 oasc.CoreContainer.load Host Name: 
   [junit4]   2> 3108430 T6438 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:52282/foo/bar
   [junit4]   2> 3108430 T6438 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 3108430 T6438 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 3108433 T6438 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3108436 T6454 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@b00f6a name:ZooKeeperConnection Watcher:127.0.0.1:52282 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3108437 T6438 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3108437 T6438 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 3108438 T6438 oascc.SolrZkClient.makePath makePath: /foo/bar
   [junit4]   2> 3108446 T6438 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 3108447 T6456 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1232469 name:ZooKeeperConnection Watcher:127.0.0.1:52282/foo/bar got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 3108447 T6438 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 3108449 T6438 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 3108453 T6438 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 3108457 T6438 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 3108462 T6438 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 3108465 T6438 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 3108470 T6438 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 3108472 T6438 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/192.168.57.16:8983_solr
   [junit4]   2> 3108473 T6438 oascc.SolrZkClient.makePath makePath: /live_nodes/192.168.57.16:8983_solr
   [junit4]   2> 3108476 T6438 oascc.SolrZkClient.makePath makePath: /collections
   [junit4]   2> 3108479 T6438 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 3108481 T6438 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 3108484 T6438 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 3108486 T6438 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 192.168.57.16:8983_solr
   [junit4]   2> 3108486 T6438 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 3108488 T6438 oasc.Overseer.start Overseer (id=92453992098758657-192.168.57.16:8983_solr-n_0000000000) starting
   [junit4]   2> 3108492 T6438 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 3108503 T6438 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 3108505 T6458 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 3108506 T6438 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 3108510 T6438 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 3108513 T6438 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 3108516 T6438 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr
   [junit4]   2> 3108518 T6457 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 3108528 T6438 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\collection1\
   [junit4]   2> 3108528 T6438 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 3108528 T6438 oasc.ZkController.bootstrapConf bootstrapping config for 1 cores into ZooKeeper using solr.xml from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\
   [junit4]   2> 3108528 T6438 oasc.ZkController.bootstrapConf Uploading directory C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\collection1\conf with name collection1 for SolrCore collection1
   [junit4]   2> 3108531 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/admin-extra.html
   [junit4]   2> 3108539 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/admin-extra.menu-bottom.html
   [junit4]   2> 3108543 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/admin-extra.menu-top.html
   [junit4]   2> 3108550 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/clustering/carrot2/kmeans-attributes.xml
   [junit4]   2> 3108557 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/clustering/carrot2/lingo-attributes.xml
   [junit4]   2> 3108563 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/clustering/carrot2/stc-attributes.xml
   [junit4]   2> 3108568 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/currency.xml
   [junit4]   2> 3108572 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/elevate.xml
   [junit4]   2> 3108576 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/contractions_ca.txt
   [junit4]   2> 3108581 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/contractions_fr.txt
   [junit4]   2> 3108586 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/contractions_ga.txt
   [junit4]   2> 3108590 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/contractions_it.txt
   [junit4]   2> 3108595 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/hyphenations_ga.txt
   [junit4]   2> 3108598 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stemdict_nl.txt
   [junit4]   2> 3108602 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stoptags_ja.txt
   [junit4]   2> 3108607 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ar.txt
   [junit4]   2> 3108611 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_bg.txt
   [junit4]   2> 3108616 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ca.txt
   [junit4]   2> 3108620 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ckb.txt
   [junit4]   2> 3108626 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_cz.txt
   [junit4]   2> 3108630 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_da.txt
   [junit4]   2> 3108634 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_de.txt
   [junit4]   2> 3108637 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_el.txt
   [junit4]   2> 3108642 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_en.txt
   [junit4]   2> 3108646 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_es.txt
   [junit4]   2> 3108650 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_eu.txt
   [junit4]   2> 3108655 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_fa.txt
   [junit4]   2> 3108658 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_fi.txt
   [junit4]   2> 3108662 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_fr.txt
   [junit4]   2> 3108668 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ga.txt
   [junit4]   2> 3108671 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_gl.txt
   [junit4]   2> 3108675 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_hi.txt
   [junit4]   2> 3108680 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_hu.txt
   [junit4]   2> 3108684 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_hy.txt
   [junit4]   2> 3108689 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_id.txt
   [junit4]   2> 3108693 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_it.txt
   [junit4]   2> 3108698 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ja.txt
   [junit4]   2> 3108703 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_lv.txt
   [junit4]   2> 3108707 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_nl.txt
   [junit4]   2> 3108712 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_no.txt
   [junit4]   2> 3108716 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_pt.txt
   [junit4]   2> 3108720 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ro.txt
   [junit4]   2> 3108725 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_ru.txt
   [junit4]   2> 3108730 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_sv.txt
   [junit4]   2> 3108734 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_th.txt
   [junit4]   2> 3108739 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/stopwords_tr.txt
   [junit4]   2> 3108743 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/lang/userdict_ja.txt
   [junit4]   2> 3108753 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/mapping-FoldToASCII.txt
   [junit4]   2> 3108759 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 3108762 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/protwords.txt
   [junit4]   2> 3108767 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/schema.xml
   [junit4]   2> 3108771 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/scripts.conf
   [junit4]   2> 3108775 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/solrconfig.xml
   [junit4]   2> 3108780 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/spellings.txt
   [junit4]   2> 3108783 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/stopwords.txt
   [junit4]   2> 3108787 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/synonyms.txt
   [junit4]   2> 3108791 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/update-script.js
   [junit4]   2> 3108795 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/browse.vm
   [junit4]   2> 3108800 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/cluster.vm
   [junit4]   2> 3108804 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/cluster_results.vm
   [junit4]   2> 3108808 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/debug.vm
   [junit4]   2> 3108811 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/did_you_mean.vm
   [junit4]   2> 3108816 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/error.vm
   [junit4]   2> 3108820 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/facets.vm
   [junit4]   2> 3108824 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/facet_fields.vm
   [junit4]   2> 3108829 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/facet_pivot.vm
   [junit4]   2> 3108833 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/facet_queries.vm
   [junit4]   2> 3108838 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/facet_ranges.vm
   [junit4]   2> 3108842 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/footer.vm
   [junit4]   2> 3108847 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/head.vm
   [junit4]   2> 3108851 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/header.vm
   [junit4]   2> 3108856 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/hit.vm
   [junit4]   2> 3108860 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/hit_grouped.vm
   [junit4]   2> 3108865 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/hit_plain.vm
   [junit4]   2> 3108870 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/join_doc.vm
   [junit4]   2> 3108873 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/jquery.autocomplete.css
   [junit4]   2> 3108879 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/jquery.autocomplete.js
   [junit4]   2> 3108883 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/layout.vm
   [junit4]   2> 3108887 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/main.css
   [junit4]   2> 3108892 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/mime_type_lists.vm
   [junit4]   2> 3108895 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/pagination_bottom.vm
   [junit4]   2> 3108900 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/pagination_top.vm
   [junit4]   2> 3108904 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/product_doc.vm
   [junit4]   2> 3108909 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/query.vm
   [junit4]   2> 3108913 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/query_form.vm
   [junit4]   2> 3108917 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/query_group.vm
   [junit4]   2> 3108920 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/query_spatial.vm
   [junit4]   2> 3108925 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/README.txt
   [junit4]   2> 3108930 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/results_list.vm
   [junit4]   2> 3108934 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/richtext_doc.vm
   [junit4]   2> 3108939 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/suggest.vm
   [junit4]   2> 3108943 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/tabs.vm
   [junit4]   2> 3108949 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/velocity/VM_global_library.vm
   [junit4]   2> 3108954 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/xslt/example.xsl
   [junit4]   2> 3108960 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/xslt/example_atom.xsl
   [junit4]   2> 3108963 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/xslt/example_rss.xsl
   [junit4]   2> 3108967 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/xslt/luke.xsl
   [junit4]   2> 3108972 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/xslt/updateXml.xsl
   [junit4]   2> 3108976 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/_rest_managed.json
   [junit4]   2> 3108980 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/_schema_analysis_stopwords_english.json
   [junit4]   2> 3108984 T6438 oascc.SolrZkClient.makePath makePath: /configs/collection1/_schema_analysis_synonyms_english.json
   [junit4]   2> 3108993 T6438 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr
   [junit4]   2> 3108996 T6438 oasc.CorePropertiesLocator.discoverUnder Found core collection1 in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\collection1\
   [junit4]   2> 3108996 T6438 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 3108998 T6460 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 3108998 T6460 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 3108999 T6460 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 3108999 T6461 oasc.DistributedQueue$LatchChildWatcher.process LatchChildWatcher fired on path: /overseer/queue state: SyncConnected type NodeChildrenChanged
   [junit4]   2> 3109003 T6457 oasc.Overseer$ClusterStateUpdater.updateState Update state numShards=null message={
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "base_url":"http://192.168.57.16:8983/solr",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "node_name":"192.168.57.16:8983_solr"}
   [junit4]   2> 3109003 T6457 oasc.Overseer$ClusterStateUpdater.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 3109008 T6461 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 3109777 T6460 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 3109777 T6460 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 3109779 T6460 oasc.ZkController.createCollectionZkNode Creating collection in ZooKeeper:collection1
   [junit4]   2> 3109779 T6460 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 3109782 T6460 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 3109783 T6460 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=collection1 specified config exists in ZooKeeper
   [junit4]   2> 3109783 T6460 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\collection1\'
   [junit4]   2> 3109811 T3 oasc.CoreContainer.finalize ERROR CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!!  instance=6371933
   [junit4]   2> 3109844 T6460 oasc.SolrConfig.initLibs Adding specified lib dirs to ClassLoader
   [junit4]   2> 3109846 T6460 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/extraction/lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\collection1\..\..\..\contrib\extraction\lib).
   [junit4]   2> 3109846 T6460 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\collection1\..\..\..\dist).
   [junit4]   2> 3109849 T6460 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/clustering/lib/ (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\collection1\..\..\..\contrib\clustering\lib).
   [junit4]   2> 3109849 T6460 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\collection1\..\..\..\dist).
   [junit4]   2> 3109849 T6460 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/langid/lib/ (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\collection1\..\..\..\contrib\langid\lib).
   [junit4]   2> 3109849 T6460 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\collection1\..\..\..\dist).
   [junit4]   2> 3109849 T6460 oasc.SolrResourceLoader.addToClassLoader WARN No files added to classloader from lib: ../../../contrib/velocity/lib (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\collection1\..\..\..\contrib\velocity\lib).
   [junit4]   2> 3109851 T6460 oasc.SolrResourceLoader.addToClassLoader WARN Can't find (or read) directory to add to classloader: ../../../dist/ (resolved as: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\example\solr\collection1\..\..\..\dist).
   [junit4]   2> 3109885 T6460 oasu.SolrIndexConfig.<init> IndexWriter infoStream solr logging is enabled
   [junit4]   2> 3109890 T6460 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 3109963 T6460 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 3109966 T6460 oass.IndexSchema.readSchema Reading Solr Schema from schema.xml
   [junit4]   2> 3109976 T6460 oass.IndexSchema.readSchema [collection1] Schema name=example
   [junit4]   2> 3110433 T6460 o

[...truncated too long message...]

.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:351)
   [junit4]    > 	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:276)
   [junit4]    > 	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:488)
   [junit4]    > 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:794)
   [junit4]    > 	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:652)
   [junit4]    > 	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:491)
   [junit4]    > 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255)
   [junit4]    > 	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249)
   [junit4]    > 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.lang.AssertionError: Directory not closed: MockDirectoryWrapper(RAMDirectory@274403 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@10ba15b)
   [junit4]    > 	at org.apache.lucene.util.CloseableDirectory.close(CloseableDirectory.java:47)
   [junit4]    > 	at com.carrotsearch.randomizedtesting.RandomizedContext.closeResources(RandomizedContext.java:183)
   [junit4]    > 	... 1 more
   [junit4] Completed in 14.64s, 4 tests, 3 errors <<< FAILURES!

[...truncated 13 lines...]
   [junit4] Suite: org.apache.solr.core.RAMDirectoryFactoryTest
   [junit4]   2> 3265593 T6986 oasc.CachingDirectoryFactory.injectLockFactory WARN No lockType configured for RAMDirectory@14b73d2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@167ece2 assuming 'simple'
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=RAMDirectoryFactoryTest -Dtests.method=test -Dtests.seed=DBD43E443B6DD2FD -Dtests.slow=true -Dtests.locale=fr_CA -Dtests.timezone=Africa/Bissau -Dtests.file.encoding=Cp1252
   [junit4] ERROR   0.01s | RAMDirectoryFactoryTest.test <<<
   [junit4]    > Throwable #1: java.security.AccessControlException: access denied ("java.io.FilePermission" "\fake\path" "write")
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DBD43E443B6DD2FD:5380019E9591BF05]:0)
   [junit4]    > 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
   [junit4]    > 	at java.security.AccessController.checkPermission(AccessController.java:559)
   [junit4]    > 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
   [junit4]    > 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979)
   [junit4]    > 	at sun.nio.fs.WindowsPath.checkWrite(WindowsPath.java:799)
   [junit4]    > 	at sun.nio.fs.WindowsFileSystemProvider.createDirectory(WindowsFileSystemProvider.java:491)
   [junit4]    > 	at java.nio.file.Files.createDirectory(Files.java:630)
   [junit4]    > 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:734)
   [junit4]    > 	at java.nio.file.Files.createDirectories(Files.java:683)
   [junit4]    > 	at org.apache.lucene.store.FSLockFactory.setLockDir(FSLockFactory.java:46)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSLockFactory.<init>(SimpleFSLockFactory.java:81)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.injectLockFactory(CachingDirectoryFactory.java:500)
   [junit4]    > 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:357)
   [junit4]    > 	at org.apache.solr.core.RAMDirectoryFactoryTest.dotestOpenReturnsTheSameForSamePath(RAMDirectoryFactoryTest.java:46)
   [junit4]    > 	at org.apache.solr.core.RAMDirectoryFactoryTest.test(RAMDirectoryFactoryTest.java:33)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=fr_CA, timezone=Africa/Bissau
   [junit4]   2> NOTE: Windows 7 6.1 x86/Oracle Corporation 1.7.0_67 (32-bit)/cpus=2,threads=1,free=231459928,total=523501568
   [junit4]   2> NOTE: All tests run in this JVM: [TestReplicationHandler, TestFreeTextSuggestions, SolrInfoMBeanTest, MinimalSchemaTest, URLClassifyProcessorTest, FunctionTest, TestArbitraryIndexDir, FieldFacetExtrasTest, TestSystemIdResolver, TestCopyFieldCollectionResource, DocumentAnalysisRequestHandlerTest, CurrencyFieldXmlFileTest, QueryParsingTest, TestRawResponseWriter, TestDynamicFieldCollectionResource, TestFastOutputStream, TestSolrXmlPersistence, IndexSchemaTest, BJQParserTest, UUIDFieldTest, TestDynamicFieldResource, TestCSVLoader, TestDistributedSearch, IndexBasedSpellCheckerTest, TestCloudManagedSchema, SolrTestCaseJ4Test, TestStressLucene, TestAnalyzedSuggestions, TestSolrDeletionPolicy1, SuggesterTSTTest, HdfsSyncSliceTest, TestSearcherReuse, NumericFieldsTest, TestOmitPositions, TestSolrXmlPersistor, TestLFUCache, TestDefaultSimilarityFactory, QueryFacetTest, TestFieldTypeResource, ShardSplitTest, RequestHandlersTest, TestManagedResourceStorage, TestFunctionQuery, AnalyticsQueryTest, DateFieldTest, UpdateRequestProcessorFactoryTest, SolrPluginUtilsTest, SpellPossibilityIteratorTest, MigrateRouteKeyTest, BadComponentTest, JsonLoaderTest, MoreLikeThisHandlerTest, TestFastWriter, ClusterStateTest, AnalysisErrorHandlingTest, CursorPagingTest, TriLevelCompositeIdRoutingTest, ShardRoutingTest, ZkCLITest, SharedFSAutoReplicaFailoverUtilsTest, TestQuerySenderListener, TestCSVResponseWriter, TestNonNRTOpen, TestMultiCoreConfBootstrap, TestQueryTypes, StatelessScriptUpdateProcessorFactoryTest, BasicDistributedZkTest, TestStressVersions, TestTrie, TestFiltering, TestUpdate, TestModifyConfFiles, LeaderElectionIntegrationTest, ConvertedLegacyTest, SpellingQueryConverterTest, TestDocumentBuilder, AssignTest, TestJmxIntegration, TestStressUserVersions, TestFaceting, MBeansHandlerTest, TestRecoveryHdfs, TestPivotHelperCode, StandardRequestHandlerTest, OpenExchangeRatesOrgProviderTest, DisMaxRequestHandlerTest, TestCoreContainer, TestExtendedDismaxParser, TestValueSourceCache, TestJoin, EchoParamsTest, SortByFunctionTest, DocValuesMultiTest, TestShortCircuitedRequests, MergeStrategyTest, TestCoreDiscovery, TestLMDirichletSimilarityFactory, ExternalCollectionsTest, HardAutoCommitTest, TestBlendedInfixSuggestions, DistributedQueryElevationComponentTest, ScriptEngineTest, TestMaxScoreQueryParser, CachingDirectoryFactoryTest, RegexBoostProcessorTest, TestRandomFaceting, DeleteInactiveReplicaTest, TestStressRecovery, TestFastLRUCache, TestExceedMaxTermLength, DateRangeFieldTest, AddBlockUpdateTest, SSLMigrationTest, ResourceLoaderTest, EnumFieldTest, UUIDUpdateProcessorFallbackTest, FieldMutatingUpdateProcessorTest, HighlighterConfigTest, BasicZkTest, TestCharFilters, TestAtomicUpdateErrorCases, SuggesterTest, TestCloudPivotFacet, TestImplicitCoreProperties, TestOverriddenPrefixQueryForCustomFieldType, StatsComponentTest, TestDistributedGrouping, TestRTGBase, TestCollationFieldDocValues, TestFoldingMultitermQuery, PolyFieldTest, CopyFieldTest, DocValuesMissingTest, TestOrdValues, TestQueryUtils, TestRangeQuery, TestParamSet, TestSolr4Spatial2, OverriddenZkACLAndCredentialsProvidersTest, DistribCursorPagingTest, BadIndexSchemaTest, AtomicUpdatesTest, DeleteReplicaTest, SearchHandlerTest, ReplicationFactorTest, CacheHeaderTest, ShowFileRequestHandlerTest, WordBreakSolrSpellCheckerTest, HdfsLockFactoryTest, BasicFunctionalityTest, TestSolrXml, FileBasedSpellCheckerTest, ChangedSchemaMergeTest, TestPerFieldSimilarity, TestNoOpRegenerator, TestLazyCores, AliasIntegrationTest, TestHighFrequencyDictionaryFactory, SimpleFacetsTest, AbstractAnalyticsStatsTest, TestIndexingPerformance, SolrCoreTest, TestSolrCoreProperties, TestBinaryField, TestBinaryResponseWriter, TestStandardQParsers, AutoCommitTest, CoreMergeIndexesAdminHandlerTest, TestCloudSchemaless, TestHashPartitioner, MultiThreadedOCPTest, SuggesterWFSTTest, NoFacetTest, DefaultValueUpdateProcessorTest, HdfsBasicDistributedZkTest, CoreAdminCreateDiscoverTest, TestSolrDeletionPolicy2, CoreAdminRequestStatusTest, FileUtilsTest, TestReloadAndDeleteDocs, CursorMarkTest, NotRequiredUniqueKeyTest, TestTolerantSearch, TestSearchPerf, TestZkChroot, TestSolrQueryParser, DistributedSpellCheckComponentTest, OverseerTest, ExpressionTest, RAMDirectoryFactoryTest]
   [junit4] Completed in 0.03s, 1 test, 1 error <<< FAILURES!

[...truncated 792 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:491: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:471: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:60: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:496: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1358: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:961: There were test failures: 438 suites, 1830 tests, 3 suite-level errors, 2 errors, 54 ignored (32 assumptions)

Total time: 173 minutes 57 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.7.0_67 -client -XX:+UseConcMarkSweepGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_20) - Build # 4311 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4311/
Java: 32bit/jdk1.8.0_20 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.benchmark.byTask.TestPerfTasksParse

Error Message:
Could not remove the following files (in the order of attempts):    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001\linefile-001.txt: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001\linefile-001.txt    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001\linefile-002.txt: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001\linefile-002.txt    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of attempts):
   C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001\linefile-001.txt: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001\linefile-001.txt
   C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001\linefile-002.txt: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001\linefile-002.txt
   C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001

	at __randomizedtesting.SeedInfo.seed([EBB0DBD1FB2CCFE9]:0)
	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288)
	at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:126)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 7125 lines...]
   [junit4] Suite: org.apache.lucene.benchmark.byTask.TestPerfTasksParse
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> Spatial Strategy: RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:26,ctx:SpatialContext{geo=true, calculator=Haversine, worldBounds=Rect(minX=-180.0,maxX=180.0,minY=-90.0,maxY=90.0)})),pointsOnly,pruneLeafyBranches)
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   1> ------------> queries:
   [junit4]   1> 
   [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=yes): {}, locale=ro_RO, timezone=Europe/Minsk
   [junit4]   2> NOTE: Windows 7 6.1 x86/Oracle Corporation 1.8.0_20 (32-bit)/cpus=2,threads=1,free=21303776,total=32571392
   [junit4]   2> NOTE: All tests run in this JVM: [TestPerfTasksParse]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestPerfTasksParse -Dtests.seed=EBB0DBD1FB2CCFE9 -Dtests.slow=true -Dtests.locale=ro_RO -Dtests.timezone=Europe/Minsk -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s | TestPerfTasksParse (suite) <<<
   [junit4]    > Throwable #1: java.io.IOException: Could not remove the following files (in the order of attempts):
   [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001\linefile-001.txt: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001\linefile-001.txt
   [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001\linefile-002.txt: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001\linefile-002.txt
   [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001: java.nio.file.DirectoryNotEmptyException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask.TestPerfTasksParse-EBB0DBD1FB2CCFE9-001
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EBB0DBD1FB2CCFE9]:0)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed in 3.21s, 3 tests, 1 error <<< FAILURES!

[...truncated 60 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:491: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:471: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:60: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build.xml:462: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:2140: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\module-build.xml:58: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1358: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:961: There were test failures: 18 suites, 93 tests, 1 suite-level error

Total time: 30 minutes 23 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.8.0_20 -server -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Re: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) - Build # 4310 - Still Failing!

Posted by Robert Muir <rc...@gmail.com>.
thank you for debugging this! I didn't know about this crazy finalizer
at all: I tried to figure it out last night, but i could not see the
problem and without a windows machine it was hopeless for me.

On Sun, Sep 14, 2014 at 7:29 AM, Uwe Schindler <uw...@thetaphi.de> wrote:
> Hi,
>
> I know why it fails consequently with the new Java 7 APIs:
> FileOutputStream since version 1.0 of Java has a custom finalizer() method, so the file gets closed on GC, kept until now for backwards compatibility. But if you the stream with Java 7's NIO.2, the created OutputStream instance (some non-public internal platform specific impl class) no longer has a finalizer(), so the file handle is never closed when the OutputStream is GCed. This is great (f*ck finalizers)!
>
> So by using NIO.2 we have a very nice "detector" for unclosed file, if people do broken stuff like "new Properties(new FileInputStream(...))" as seen in Solr quite often!
>
> Uwe
>
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
>
>
>> -----Original Message-----
>> From: Uwe Schindler [mailto:uwe@thetaphi.de]
>> Sent: Sunday, September 14, 2014 1:16 PM
>> To: dev@lucene.apache.org
>> Subject: RE: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) -
>> Build # 4310 - Still Failing!
>>
>> The problem here is incomplete cleanup:
>>
>> This test parses the whole algorithm, it then creates an Algorithm instance
>> containing all Tasks to execute. The "WriteLineDocsTask" is instantiated,
>> which in fact opens the output stream for the line docs file in its ctor. But as
>> the tasks are never executed, so the Task's close() is never called.
>>
>> I fixed the whole thing by making Algorithm implement AutoCloseable (not
>> Closeable, because it throws not only IOException). I also backported the
>> whole stuff to Lucene 4, because this is unrelated to Robert's changes.
>> Roberts changes just triggered other code execution paths, so GC did not
>> clean up algorithm.
>>
>> TODO: verify other uses of Algorithm and check that they are correctly
>> closed. In fact I have seen this bug sometimes when running tests locally, so
>> this was existent since long time, just in most cases GC was faster!
>>
>> Uwe
>>
>> -----
>> Uwe Schindler
>> H.-H.-Meier-Allee 63, D-28213 Bremen
>> http://www.thetaphi.de
>> eMail: uwe@thetaphi.de
>>
>>
>> > -----Original Message-----
>> > From: Uwe Schindler [mailto:uwe@thetaphi.de]
>> > Sent: Sunday, September 14, 2014 12:21 PM
>> > To: dev@lucene.apache.org
>> > Subject: RE: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) -
>> > Build # 4310 - Still Failing!
>> >
>> > This fails reproducible.
>> >
>> > I will dig!
>> >
>> > -----
>> > Uwe Schindler
>> > H.-H.-Meier-Allee 63, D-28213 Bremen
>> > http://www.thetaphi.de
>> > eMail: uwe@thetaphi.de
>> >
>> > > -----Original Message-----
>> > > From: Policeman Jenkins Server [mailto:jenkins@thetaphi.de]
>> > > Sent: Sunday, September 14, 2014 2:48 AM
>> > > To: noble@apache.org; sarowe@gmail.com; rmuir@apache.org;
>> > > uwe@thetaphi.de; mikemccand@apache.org; dev@lucene.apache.org
>> > > Subject: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) -
>> > > Build #
>> > > 4310 - Still Failing!
>> > >
>> > > Build:
>> > > http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4310/
>> > > Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops -
>> > > XX:+UseConcMarkSweepGC
>> > >
>> > > 1 tests failed.
>> > > FAILED:
>> > >
>> >
>> junit.framework.TestSuite.org.apache.lucene.benchmark.byTask.TestPerfT
>> > > a
>> > > sksParse
>> > >
>> > > Error Message:
>> > > Could not remove the following files (in the order of attempts):
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt:
>> > > java.nio.file.AccessDeniedException:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt:
>> > > java.nio.file.AccessDeniedException:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001:
>> > > java.nio.file.DirectoryNotEmptyException:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001
>> > >
>> > > Stack Trace:
>> > > java.io.IOException: Could not remove the following files (in the
>> > > order of
>> > > attempts):
>> > >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt:
>> > > java.nio.file.AccessDeniedException:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
>> > >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt:
>> > > java.nio.file.AccessDeniedException:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
>> > >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001:
>> > > java.nio.file.DirectoryNotEmptyException:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001
>> > >
>> > >   at __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0)
>> > >   at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288)
>> > >   at
>> > > org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(Tes
>> > > tR
>> > > ul
>> > > eTemporaryFilesCleanup.java:126)
>> > >   at
>> > > com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlwa
>> > > ys
>> > > (T
>> > > estRuleAdapter.java:31)
>> > >   at
>> > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(S
>> > > ta
>> > > t
>> > > ementAdapter.java:43)
>> > >   at
>> > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(S
>> > > ta
>> > > t
>> > > ementAdapter.java:36)
>> > >   at
>> > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRul
>> > > eA
>> > > ss
>> > > ertionsRequired.java:43)
>> > >   at
>> > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFa
>> > > il
>> > > ure
>> > > .java:48)
>> > >   at
>> > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(Tes
>> > > tR
>> > > ule
>> > > IgnoreAfterMaxFailures.java:65)
>> > >   at
>> > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleI
>> > > gn
>> > > ore
>> > > TestSuites.java:55)
>> > >   at
>> > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(S
>> > > ta
>> > > t
>> > > ementAdapter.java:36)
>> > >   at
>> > >
>> >
>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.
>> > > run(ThreadLeakControl.java:365)
>> > >   at java.lang.Thread.run(Thread.java:745)
>> > >
>> > >
>> > >
>> > >
>> > > Build Log:
>> > > [...truncated 7622 lines...]
>> > >    [junit4] Suite:
>> org.apache.lucene.benchmark.byTask.TestPerfTasksParse
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> Spatial Strategy:
>> > > RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:26,ctx:Spa
>> > > ti al Context{geo=true, calculator=Haversine,
>> > > worldBounds=Rect(minX=-
>> > > 180.0,maxX=180.0,minY=-
>> > > 90.0,maxY=90.0)})),pointsOnly,pruneLeafyBranches)
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   1> ------------> queries:
>> > >    [junit4]   1>
>> > >    [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{},
>> > > sim=DefaultSimilarity, locale=sq_AL, timezone=America/Yakutat
>> > >    [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_67
>> (64-
>> > > bit)/cpus=2,threads=1,free=120281832,total=140083200
>> > >    [junit4]   2> NOTE: All tests run in this JVM: [DocMakerTest,
>> > > WriteLineDocTaskTest, AddIndexesTaskTest, StreamUtilsTest,
>> > > TrecContentSourceTest, TestQualityRun, PerfTaskTest,
>> > > LineDocSourceTest, TestPerfTasksLogic, TestPerfTasksParse]
>> > >    [junit4]   2> NOTE: reproduce with: ant test  -
>> > Dtestcase=TestPerfTasksParse
>> > > -Dtests.seed=2D34950549928D3F -Dtests.slow=true -Dtests.locale=sq_AL
>> > > - Dtests.timezone=America/Yakutat -Dtests.file.encoding=ISO-8859-1
>> > >    [junit4] ERROR   0.00s | TestPerfTasksParse (suite) <<<
>> > >    [junit4]    > Throwable #1: java.io.IOException: Could not remove the
>> > > following files (in the order of attempts):
>> > >    [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt:
>> > > java.nio.file.AccessDeniedException:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
>> > >    [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt:
>> > > java.nio.file.AccessDeniedException:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
>> > >    [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001:
>> > > java.nio.file.DirectoryNotEmptyException:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > >
>> >
>> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
>> > > .TestPerfTasksParse-2D34950549928D3F-001
>> > >    [junit4]    >  at
>> > > __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0)
>> > >    [junit4]    >  at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288)
>> > >    [junit4]    >  at java.lang.Thread.run(Thread.java:745)
>> > >    [junit4] Completed in 3.00s, 3 tests, 1 error <<< FAILURES!
>> > >
>> > > [...truncated 33 lines...]
>> > > BUILD FAILED
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > > Windows\build.xml:491: The following error occurred while executing
>> > > this
>> > > line:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > > Windows\build.xml:471: The following error occurred while executing
>> > > this
>> > > line:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > Windows\build.xml:60:
>> > > The following error occurred while executing this line:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-
>> > > targets.xml:39: The following error occurred while executing this line:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > > Windows\lucene\build.xml:462: The following error occurred while
>> > > executing this line:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > > Windows\lucene\common-build.xml:2140: The following error occurred
>> > > while executing this line:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > > Windows\lucene\module-build.xml:58: The following error occurred
>> > > while executing this line:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > > Windows\lucene\common-build.xml:1358: The following error occurred
>> > > while executing this line:
>> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
>> > > Windows\lucene\common-build.xml:961: There were test failures: 18
>> > > suites,
>> > > 93 tests, 1 suite-level error
>> > >
>> > > Total time: 31 minutes 50 seconds
>> > > Build step 'Invoke Ant' marked build as failure [description-setter]
>> > > Description set: Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops -
>> > > XX:+UseConcMarkSweepGC Archiving artifacts Recording test results
>> > > Email was triggered for: Failure - Any Sending email for trigger:
>> > > Failure - Any
>> > >
>> >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
>> > additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
>> commands, e-mail: dev-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


RE: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) - Build # 4310 - Still Failing!

Posted by Uwe Schindler <uw...@thetaphi.de>.
Hi,

I know why it fails consequently with the new Java 7 APIs:
FileOutputStream since version 1.0 of Java has a custom finalizer() method, so the file gets closed on GC, kept until now for backwards compatibility. But if you the stream with Java 7's NIO.2, the created OutputStream instance (some non-public internal platform specific impl class) no longer has a finalizer(), so the file handle is never closed when the OutputStream is GCed. This is great (f*ck finalizers)!

So by using NIO.2 we have a very nice "detector" for unclosed file, if people do broken stuff like "new Properties(new FileInputStream(...))" as seen in Solr quite often!

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> Sent: Sunday, September 14, 2014 1:16 PM
> To: dev@lucene.apache.org
> Subject: RE: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) -
> Build # 4310 - Still Failing!
> 
> The problem here is incomplete cleanup:
> 
> This test parses the whole algorithm, it then creates an Algorithm instance
> containing all Tasks to execute. The "WriteLineDocsTask" is instantiated,
> which in fact opens the output stream for the line docs file in its ctor. But as
> the tasks are never executed, so the Task's close() is never called.
> 
> I fixed the whole thing by making Algorithm implement AutoCloseable (not
> Closeable, because it throws not only IOException). I also backported the
> whole stuff to Lucene 4, because this is unrelated to Robert's changes.
> Roberts changes just triggered other code execution paths, so GC did not
> clean up algorithm.
> 
> TODO: verify other uses of Algorithm and check that they are correctly
> closed. In fact I have seen this bug sometimes when running tests locally, so
> this was existent since long time, just in most cases GC was faster!
> 
> Uwe
> 
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
> 
> 
> > -----Original Message-----
> > From: Uwe Schindler [mailto:uwe@thetaphi.de]
> > Sent: Sunday, September 14, 2014 12:21 PM
> > To: dev@lucene.apache.org
> > Subject: RE: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) -
> > Build # 4310 - Still Failing!
> >
> > This fails reproducible.
> >
> > I will dig!
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: uwe@thetaphi.de
> >
> > > -----Original Message-----
> > > From: Policeman Jenkins Server [mailto:jenkins@thetaphi.de]
> > > Sent: Sunday, September 14, 2014 2:48 AM
> > > To: noble@apache.org; sarowe@gmail.com; rmuir@apache.org;
> > > uwe@thetaphi.de; mikemccand@apache.org; dev@lucene.apache.org
> > > Subject: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) -
> > > Build #
> > > 4310 - Still Failing!
> > >
> > > Build:
> > > http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4310/
> > > Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops -
> > > XX:+UseConcMarkSweepGC
> > >
> > > 1 tests failed.
> > > FAILED:
> > >
> >
> junit.framework.TestSuite.org.apache.lucene.benchmark.byTask.TestPerfT
> > > a
> > > sksParse
> > >
> > > Error Message:
> > > Could not remove the following files (in the order of attempts):
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt:
> > > java.nio.file.AccessDeniedException:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt:
> > > java.nio.file.AccessDeniedException:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001:
> > > java.nio.file.DirectoryNotEmptyException:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001
> > >
> > > Stack Trace:
> > > java.io.IOException: Could not remove the following files (in the
> > > order of
> > > attempts):
> > >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt:
> > > java.nio.file.AccessDeniedException:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
> > >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt:
> > > java.nio.file.AccessDeniedException:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
> > >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001:
> > > java.nio.file.DirectoryNotEmptyException:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001
> > >
> > > 	at __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0)
> > > 	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288)
> > > 	at
> > > org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(Tes
> > > tR
> > > ul
> > > eTemporaryFilesCleanup.java:126)
> > > 	at
> > > com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlwa
> > > ys
> > > (T
> > > estRuleAdapter.java:31)
> > > 	at
> > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(S
> > > ta
> > > t
> > > ementAdapter.java:43)
> > > 	at
> > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(S
> > > ta
> > > t
> > > ementAdapter.java:36)
> > > 	at
> > > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRul
> > > eA
> > > ss
> > > ertionsRequired.java:43)
> > > 	at
> > > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFa
> > > il
> > > ure
> > > .java:48)
> > > 	at
> > > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(Tes
> > > tR
> > > ule
> > > IgnoreAfterMaxFailures.java:65)
> > > 	at
> > > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleI
> > > gn
> > > ore
> > > TestSuites.java:55)
> > > 	at
> > > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(S
> > > ta
> > > t
> > > ementAdapter.java:36)
> > > 	at
> > >
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.
> > > run(ThreadLeakControl.java:365)
> > > 	at java.lang.Thread.run(Thread.java:745)
> > >
> > >
> > >
> > >
> > > Build Log:
> > > [...truncated 7622 lines...]
> > >    [junit4] Suite:
> org.apache.lucene.benchmark.byTask.TestPerfTasksParse
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> Spatial Strategy:
> > > RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:26,ctx:Spa
> > > ti al Context{geo=true, calculator=Haversine,
> > > worldBounds=Rect(minX=-
> > > 180.0,maxX=180.0,minY=-
> > > 90.0,maxY=90.0)})),pointsOnly,pruneLeafyBranches)
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   1> ------------> queries:
> > >    [junit4]   1>
> > >    [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{},
> > > sim=DefaultSimilarity, locale=sq_AL, timezone=America/Yakutat
> > >    [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_67
> (64-
> > > bit)/cpus=2,threads=1,free=120281832,total=140083200
> > >    [junit4]   2> NOTE: All tests run in this JVM: [DocMakerTest,
> > > WriteLineDocTaskTest, AddIndexesTaskTest, StreamUtilsTest,
> > > TrecContentSourceTest, TestQualityRun, PerfTaskTest,
> > > LineDocSourceTest, TestPerfTasksLogic, TestPerfTasksParse]
> > >    [junit4]   2> NOTE: reproduce with: ant test  -
> > Dtestcase=TestPerfTasksParse
> > > -Dtests.seed=2D34950549928D3F -Dtests.slow=true -Dtests.locale=sq_AL
> > > - Dtests.timezone=America/Yakutat -Dtests.file.encoding=ISO-8859-1
> > >    [junit4] ERROR   0.00s | TestPerfTasksParse (suite) <<<
> > >    [junit4]    > Throwable #1: java.io.IOException: Could not remove the
> > > following files (in the order of attempts):
> > >    [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt:
> > > java.nio.file.AccessDeniedException:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
> > >    [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt:
> > > java.nio.file.AccessDeniedException:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
> > >    [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001:
> > > java.nio.file.DirectoryNotEmptyException:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > >
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > > .TestPerfTasksParse-2D34950549928D3F-001
> > >    [junit4]    > 	at
> > > __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0)
> > >    [junit4]    > 	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288)
> > >    [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
> > >    [junit4] Completed in 3.00s, 3 tests, 1 error <<< FAILURES!
> > >
> > > [...truncated 33 lines...]
> > > BUILD FAILED
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > > Windows\build.xml:491: The following error occurred while executing
> > > this
> > > line:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > > Windows\build.xml:471: The following error occurred while executing
> > > this
> > > line:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > Windows\build.xml:60:
> > > The following error occurred while executing this line:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-
> > > targets.xml:39: The following error occurred while executing this line:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > > Windows\lucene\build.xml:462: The following error occurred while
> > > executing this line:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > > Windows\lucene\common-build.xml:2140: The following error occurred
> > > while executing this line:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > > Windows\lucene\module-build.xml:58: The following error occurred
> > > while executing this line:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > > Windows\lucene\common-build.xml:1358: The following error occurred
> > > while executing this line:
> > > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > > Windows\lucene\common-build.xml:961: There were test failures: 18
> > > suites,
> > > 93 tests, 1 suite-level error
> > >
> > > Total time: 31 minutes 50 seconds
> > > Build step 'Invoke Ant' marked build as failure [description-setter]
> > > Description set: Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops -
> > > XX:+UseConcMarkSweepGC Archiving artifacts Recording test results
> > > Email was triggered for: Failure - Any Sending email for trigger:
> > > Failure - Any
> > >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For
> > additional commands, e-mail: dev-help@lucene.apache.org
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


RE: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) - Build # 4310 - Still Failing!

Posted by Uwe Schindler <uw...@thetaphi.de>.
The problem here is incomplete cleanup:

This test parses the whole algorithm, it then creates an Algorithm instance containing all Tasks to execute. The "WriteLineDocsTask" is instantiated, which in fact opens the output stream for the line docs file in its ctor. But as the tasks are never executed, so the Task's close() is never called.

I fixed the whole thing by making Algorithm implement AutoCloseable (not Closeable, because it throws not only IOException). I also backported the whole stuff to Lucene 4, because this is unrelated to Robert's changes. Roberts changes just triggered other code execution paths, so GC did not clean up algorithm.

TODO: verify other uses of Algorithm and check that they are correctly closed. In fact I have seen this bug sometimes when running tests locally, so this was existent since long time, just in most cases GC was faster!

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Uwe Schindler [mailto:uwe@thetaphi.de]
> Sent: Sunday, September 14, 2014 12:21 PM
> To: dev@lucene.apache.org
> Subject: RE: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) -
> Build # 4310 - Still Failing!
> 
> This fails reproducible.
> 
> I will dig!
> 
> -----
> Uwe Schindler
> H.-H.-Meier-Allee 63, D-28213 Bremen
> http://www.thetaphi.de
> eMail: uwe@thetaphi.de
> 
> > -----Original Message-----
> > From: Policeman Jenkins Server [mailto:jenkins@thetaphi.de]
> > Sent: Sunday, September 14, 2014 2:48 AM
> > To: noble@apache.org; sarowe@gmail.com; rmuir@apache.org;
> > uwe@thetaphi.de; mikemccand@apache.org; dev@lucene.apache.org
> > Subject: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) -
> > Build #
> > 4310 - Still Failing!
> >
> > Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4310/
> > Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops -
> > XX:+UseConcMarkSweepGC
> >
> > 1 tests failed.
> > FAILED:
> >
> junit.framework.TestSuite.org.apache.lucene.benchmark.byTask.TestPerfT
> > a
> > sksParse
> >
> > Error Message:
> > Could not remove the following files (in the order of attempts):
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt:
> > java.nio.file.AccessDeniedException:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt:
> > java.nio.file.AccessDeniedException:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001:
> > java.nio.file.DirectoryNotEmptyException:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001
> >
> > Stack Trace:
> > java.io.IOException: Could not remove the following files (in the
> > order of
> > attempts):
> >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt:
> > java.nio.file.AccessDeniedException:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
> >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt:
> > java.nio.file.AccessDeniedException:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
> >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001:
> > java.nio.file.DirectoryNotEmptyException:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001
> >
> > 	at __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0)
> > 	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288)
> > 	at
> > org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestR
> > ul
> > eTemporaryFilesCleanup.java:126)
> > 	at
> > com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways
> > (T
> > estRuleAdapter.java:31)
> > 	at
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Sta
> > t
> > ementAdapter.java:43)
> > 	at
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Sta
> > t
> > ementAdapter.java:36)
> > 	at
> > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleA
> > ss
> > ertionsRequired.java:43)
> > 	at
> > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFail
> > ure
> > .java:48)
> > 	at
> > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestR
> > ule
> > IgnoreAfterMaxFailures.java:65)
> > 	at
> > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgn
> > ore
> > TestSuites.java:55)
> > 	at
> > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Sta
> > t
> > ementAdapter.java:36)
> > 	at
> >
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.
> > run(ThreadLeakControl.java:365)
> > 	at java.lang.Thread.run(Thread.java:745)
> >
> >
> >
> >
> > Build Log:
> > [...truncated 7622 lines...]
> >    [junit4] Suite: org.apache.lucene.benchmark.byTask.TestPerfTasksParse
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> Spatial Strategy:
> > RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:26,ctx:Spati
> > al Context{geo=true, calculator=Haversine, worldBounds=Rect(minX=-
> > 180.0,maxX=180.0,minY=-
> > 90.0,maxY=90.0)})),pointsOnly,pruneLeafyBranches)
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   1> ------------> queries:
> >    [junit4]   1>
> >    [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{},
> > sim=DefaultSimilarity, locale=sq_AL, timezone=America/Yakutat
> >    [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_67 (64-
> > bit)/cpus=2,threads=1,free=120281832,total=140083200
> >    [junit4]   2> NOTE: All tests run in this JVM: [DocMakerTest,
> > WriteLineDocTaskTest, AddIndexesTaskTest, StreamUtilsTest,
> > TrecContentSourceTest, TestQualityRun, PerfTaskTest,
> > LineDocSourceTest, TestPerfTasksLogic, TestPerfTasksParse]
> >    [junit4]   2> NOTE: reproduce with: ant test  -
> Dtestcase=TestPerfTasksParse
> > -Dtests.seed=2D34950549928D3F -Dtests.slow=true -Dtests.locale=sq_AL -
> > Dtests.timezone=America/Yakutat -Dtests.file.encoding=ISO-8859-1
> >    [junit4] ERROR   0.00s | TestPerfTasksParse (suite) <<<
> >    [junit4]    > Throwable #1: java.io.IOException: Could not remove the
> > following files (in the order of attempts):
> >    [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt:
> > java.nio.file.AccessDeniedException:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
> >    [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt:
> > java.nio.file.AccessDeniedException:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
> >    [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001:
> > java.nio.file.DirectoryNotEmptyException:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> >
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> > .TestPerfTasksParse-2D34950549928D3F-001
> >    [junit4]    > 	at
> > __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0)
> >    [junit4]    > 	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288)
> >    [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
> >    [junit4] Completed in 3.00s, 3 tests, 1 error <<< FAILURES!
> >
> > [...truncated 33 lines...]
> > BUILD FAILED
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > Windows\build.xml:491: The following error occurred while executing
> > this
> > line:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > Windows\build.xml:471: The following error occurred while executing
> > this
> > line:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\build.xml:60:
> > The following error occurred while executing this line:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-
> > targets.xml:39: The following error occurred while executing this line:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > Windows\lucene\build.xml:462: The following error occurred while
> > executing this line:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > Windows\lucene\common-build.xml:2140: The following error occurred
> > while executing this line:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > Windows\lucene\module-build.xml:58: The following error occurred while
> > executing this line:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > Windows\lucene\common-build.xml:1358: The following error occurred
> > while executing this line:
> > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> > Windows\lucene\common-build.xml:961: There were test failures: 18
> > suites,
> > 93 tests, 1 suite-level error
> >
> > Total time: 31 minutes 50 seconds
> > Build step 'Invoke Ant' marked build as failure [description-setter]
> > Description set: Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops -
> > XX:+UseConcMarkSweepGC Archiving artifacts Recording test results
> > Email was triggered for: Failure - Any Sending email for trigger:
> > Failure - Any
> >
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional
> commands, e-mail: dev-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


RE: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) - Build # 4310 - Still Failing!

Posted by Uwe Schindler <uw...@thetaphi.de>.
This fails reproducible.

I will dig!

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de

> -----Original Message-----
> From: Policeman Jenkins Server [mailto:jenkins@thetaphi.de]
> Sent: Sunday, September 14, 2014 2:48 AM
> To: noble@apache.org; sarowe@gmail.com; rmuir@apache.org;
> uwe@thetaphi.de; mikemccand@apache.org; dev@lucene.apache.org
> Subject: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.7.0_67) - Build #
> 4310 - Still Failing!
> 
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4310/
> Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops -
> XX:+UseConcMarkSweepGC
> 
> 1 tests failed.
> FAILED:
> junit.framework.TestSuite.org.apache.lucene.benchmark.byTask.TestPerfTa
> sksParse
> 
> Error Message:
> Could not remove the following files (in the order of attempts):
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt:
> java.nio.file.AccessDeniedException:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt:
> java.nio.file.AccessDeniedException:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001:
> java.nio.file.DirectoryNotEmptyException:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001
> 
> Stack Trace:
> java.io.IOException: Could not remove the following files (in the order of
> attempts):
>    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt:
> java.nio.file.AccessDeniedException:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
>    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt:
> java.nio.file.AccessDeniedException:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
>    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001:
> java.nio.file.DirectoryNotEmptyException:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001
> 
> 	at __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0)
> 	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288)
> 	at
> org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRul
> eTemporaryFilesCleanup.java:126)
> 	at
> com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(T
> estRuleAdapter.java:31)
> 	at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
> ementAdapter.java:43)
> 	at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
> ementAdapter.java:36)
> 	at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAss
> ertionsRequired.java:43)
> 	at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure
> .java:48)
> 	at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRule
> IgnoreAfterMaxFailures.java:65)
> 	at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnore
> TestSuites.java:55)
> 	at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(Stat
> ementAdapter.java:36)
> 	at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.
> run(ThreadLeakControl.java:365)
> 	at java.lang.Thread.run(Thread.java:745)
> 
> 
> 
> 
> Build Log:
> [...truncated 7622 lines...]
>    [junit4] Suite: org.apache.lucene.benchmark.byTask.TestPerfTasksParse
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> Spatial Strategy:
> RecursivePrefixTreeStrategy(SPG:(QuadPrefixTree(maxLevels:26,ctx:Spatial
> Context{geo=true, calculator=Haversine, worldBounds=Rect(minX=-
> 180.0,maxX=180.0,minY=-
> 90.0,maxY=90.0)})),pointsOnly,pruneLeafyBranches)
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   1> ------------> queries:
>    [junit4]   1>
>    [junit4]   2> NOTE: test params are: codec=Lucene410: {}, docValues:{},
> sim=DefaultSimilarity, locale=sq_AL, timezone=America/Yakutat
>    [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.7.0_67 (64-
> bit)/cpus=2,threads=1,free=120281832,total=140083200
>    [junit4]   2> NOTE: All tests run in this JVM: [DocMakerTest,
> WriteLineDocTaskTest, AddIndexesTaskTest, StreamUtilsTest,
> TrecContentSourceTest, TestQualityRun, PerfTaskTest, LineDocSourceTest,
> TestPerfTasksLogic, TestPerfTasksParse]
>    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestPerfTasksParse
> -Dtests.seed=2D34950549928D3F -Dtests.slow=true -Dtests.locale=sq_AL -
> Dtests.timezone=America/Yakutat -Dtests.file.encoding=ISO-8859-1
>    [junit4] ERROR   0.00s | TestPerfTasksParse (suite) <<<
>    [junit4]    > Throwable #1: java.io.IOException: Could not remove the
> following files (in the order of attempts):
>    [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt:
> java.nio.file.AccessDeniedException:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001\linefile-001.txt
>    [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt:
> java.nio.file.AccessDeniedException:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001\linefile-002.txt
>    [junit4]    >    C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001:
> java.nio.file.DirectoryNotEmptyException:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build\benchmark\test\J0\temp\lucene.benchmark.byTask
> .TestPerfTasksParse-2D34950549928D3F-001
>    [junit4]    > 	at
> __randomizedtesting.SeedInfo.seed([2D34950549928D3F]:0)
>    [junit4]    > 	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:288)
>    [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
>    [junit4] Completed in 3.00s, 3 tests, 1 error <<< FAILURES!
> 
> [...truncated 33 lines...]
> BUILD FAILED
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\build.xml:491: The following error occurred while executing this
> line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\build.xml:471: The following error occurred while executing this
> line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:60:
> The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-
> targets.xml:39: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\build.xml:462: The following error occurred while executing
> this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\common-build.xml:2140: The following error occurred
> while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\module-build.xml:58: The following error occurred while
> executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\common-build.xml:1358: The following error occurred
> while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-
> Windows\lucene\common-build.xml:961: There were test failures: 18 suites,
> 93 tests, 1 suite-level error
> 
> Total time: 31 minutes 50 seconds
> Build step 'Invoke Ant' marked build as failure [description-setter]
> Description set: Java: 64bit/jdk1.7.0_67 -XX:-UseCompressedOops -
> XX:+UseConcMarkSweepGC Archiving artifacts Recording test results Email
> was triggered for: Failure - Any Sending email for trigger: Failure - Any
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org