You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2016/07/28 18:41:00 UTC

[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 129 - Still Unstable

Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/129/

5 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:50211/_/hb/forceleader_test_collection_shard1_replica3]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live SolrServers available to handle this request:[http://127.0.0.1:50211/_/hb/forceleader_test_collection_shard1_replica3]
	at __randomizedtesting.SeedInfo.seed([EA7083120FCEF20D:CE7B7D2364C0B6C]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
	at org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
	at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:131)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:50211/_/hb/forceleader_test_collection_shard1_replica3]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:735)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more
Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:50211/_/hb/forceleader_test_collection_shard1_replica3
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:597)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:372)
	... 6 more
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 10 more


FAILED:  org.apache.solr.core.OpenCloseCoreStressTest.test10Minutes

Error Message:
Captured an uncaught exception in thread: Thread[id=11598, name=Lucene Merge Thread #5, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11598, name=Lucene Merge Thread #5, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest]
Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot merge
	at __randomizedtesting.SeedInfo.seed([EA7083120FCEF20D]:0)
	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
Caused by: java.lang.IllegalStateException: this writer hit an unrecoverable error; cannot merge
	at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4046)
	at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:4026)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3880)
	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_EA7083120FCEF20D-001/index-SimpleFSDirectory-004/_4b_TestBloomFilteredLucenePostings_0.tip: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
	at org.apache.lucene.mockfile.HandleTrackingFS.newByteChannel(HandleTrackingFS.java:271)
	at java.nio.file.Files.newByteChannel(Files.java:361)
	at java.nio.file.Files.newByteChannel(Files.java:407)
	at org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:77)
	at org.apache.lucene.util.LuceneTestCase.slowFileExists(LuceneTestCase.java:2695)
	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:749)
	at org.apache.lucene.codecs.blocktree.BlockTreeTermsReader.<init>(BlockTreeTermsReader.java:176)
	at org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat.fieldsProducer(Lucene50PostingsFormat.java:445)
	at org.apache.lucene.codecs.bloom.BloomFilteringPostingsFormat$BloomFilteredFieldsProducer.<init>(BloomFilteringPostingsFormat.java:174)
	at org.apache.lucene.codecs.bloom.BloomFilteringPostingsFormat.fieldsProducer(BloomFilteringPostingsFormat.java:150)
	at org.apache.lucene.codecs.bloom.TestBloomFilteredLucenePostings.fieldsProducer(TestBloomFilteredLucenePostings.java:71)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:106)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:74)
	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:384)
	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:416)
	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
	at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4068)
	... 4 more


FAILED:  org.apache.solr.core.OpenCloseCoreStressTest.test15Seconds

Error Message:
Captured an uncaught exception in thread: Thread[id=11830, name=Lucene Merge Thread #11, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11830, name=Lucene Merge Thread #11, state=RUNNABLE, group=TGRP-OpenCloseCoreStressTest]
Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_EA7083120FCEF20D-001/index-SimpleFSDirectory-014/_ap_LuceneFixedGap_0.tii: Too many open files
	at __randomizedtesting.SeedInfo.seed([EA7083120FCEF20D]:0)
	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648)
Caused by: java.nio.file.FileSystemException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.core.OpenCloseCoreStressTest_EA7083120FCEF20D-001/index-SimpleFSDirectory-014/_ap_LuceneFixedGap_0.tii: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
	at org.apache.lucene.mockfile.HandleTrackingFS.newByteChannel(HandleTrackingFS.java:271)
	at java.nio.file.Files.newByteChannel(Files.java:361)
	at java.nio.file.Files.newByteChannel(Files.java:407)
	at org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:77)
	at org.apache.lucene.util.LuceneTestCase.slowFileExists(LuceneTestCase.java:2695)
	at org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:749)
	at org.apache.lucene.codecs.blockterms.FixedGapTermsIndexReader.<init>(FixedGapTermsIndexReader.java:71)
	at org.apache.lucene.codecs.blockterms.LuceneFixedGap.fieldsProducer(LuceneFixedGap.java:101)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat$FieldsReader.<init>(PerFieldPostingsFormat.java:261)
	at org.apache.lucene.codecs.perfield.PerFieldPostingsFormat.fieldsProducer(PerFieldPostingsFormat.java:341)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:106)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:74)
	at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:145)
	at org.apache.lucene.index.BufferedUpdatesStream$SegmentState.<init>(BufferedUpdatesStream.java:384)
	at org.apache.lucene.index.BufferedUpdatesStream.openSegmentStates(BufferedUpdatesStream.java:416)
	at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:261)
	at org.apache.lucene.index.IndexWriter._mergeInit(IndexWriter.java:4068)
	at org.apache.lucene.index.IndexWriter.mergeInit(IndexWriter.java:4026)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3880)
	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([EA7083120FCEF20D]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=7766, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=7766, name=collection2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:40243: collection already exists: awholynewstresscollection_collection2_1
	at __randomizedtesting.SeedInfo.seed([EA7083120FCEF20D]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)




Build Log:
[...truncated 11378 lines...]
   [junit4] Suite: org.apache.solr.cloud.ForceLeaderTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/init-core-data-001
   [junit4]   2> 472255 INFO  (SUITE-ForceLeaderTest-seed#[EA7083120FCEF20D]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 472255 INFO  (SUITE-ForceLeaderTest-seed#[EA7083120FCEF20D]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_/hb
   [junit4]   2> 472256 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 472279 INFO  (Thread-2752) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 472279 INFO  (Thread-2752) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 472379 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.ZkTestServer start zk server on port:42388
   [junit4]   2> 472379 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 472398 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 472507 INFO  (zkCallback-700-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@19218242 name:ZooKeeperConnection Watcher:127.0.0.1:42388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 472507 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 472507 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 472507 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 472605 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 472617 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 472661 INFO  (zkCallback-701-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3d343705 name:ZooKeeperConnection Watcher:127.0.0.1:42388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 472661 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 472661 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 472661 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 472662 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 472662 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 472663 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 472663 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 472664 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 472665 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 472665 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 472666 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 472666 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 472666 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 472667 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 472667 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 472667 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 472668 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 472668 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 472668 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 472668 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 472669 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 472669 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 472670 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 472670 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 472670 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 472670 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 472671 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 472671 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 472748 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1
   [junit4]   2> 472752 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 472753 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6880e69c{/_/hb,null,AVAILABLE}
   [junit4]   2> 472759 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1d08414f{HTTP/1.1,[http/1.1]}{127.0.0.1:41555}
   [junit4]   2> 472759 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.Server Started @475042ms
   [junit4]   2> 472759 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/tempDir-001/control/data, hostContext=/_/hb, hostPort=45193, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores}
   [junit4]   2> 472763 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 472763 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001'
   [junit4]   2> 472763 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 472763 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 472763 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 472764 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 472766 INFO  (zkCallback-702-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7e79c4a name:ZooKeeperConnection Watcher:127.0.0.1:42388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 472768 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 472768 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 472769 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 472769 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/solr.xml
   [junit4]   2> 472773 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores
   [junit4]   2> 472773 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.CoreContainer New CoreContainer 1885481281
   [junit4]   2> 472774 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001]
   [junit4]   2> 472774 WARN  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/lib
   [junit4]   2> 472774 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 472775 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 472776 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 472776 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 472776 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42388/solr
   [junit4]   2> 472776 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 472776 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 472776 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 472777 INFO  (zkCallback-705-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5efd3b96 name:ZooKeeperConnection Watcher:127.0.0.1:42388 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 472777 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 472777 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 472778 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$59/1050007703@56e0da71
   [junit4]   2> 472778 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 472779 INFO  (zkCallback-706-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7b7e58f4 name:ZooKeeperConnection Watcher:127.0.0.1:42388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 472779 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 472780 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 472780 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 472781 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 472782 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 472783 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 472783 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 472784 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 472784 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 472784 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 472785 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 472785 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 472785 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 472786 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 472787 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 472787 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 472787 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000
   [junit4]   2> 472787 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45193__%2Fhb
   [junit4]   2> 472787 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 472788 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer Overseer (id=96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) starting
   [junit4]   2> 472788 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 472791 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 472791 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45193__%2Fhb
   [junit4]   2> 472791 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:45193__%2Fhb
   [junit4]   2> 472798 INFO  (zkCallback-706-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 472798 INFO  (zkCallback-706-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 472798 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 472798 INFO  (OverseerCollectionConfigSetProcessor-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 472799 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 472805 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 472805 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 472805 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores
   [junit4]   2> 472806 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 472806 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1
   [junit4]   2> 472806 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 472808 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 472808 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 472809 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 472809 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45193/_/hb",
   [junit4]   2>   "node_name":"127.0.0.1:45193__%2Fhb",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 472809 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 472809 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45193/_/hb",
   [junit4]   2>   "node_name":"127.0.0.1:45193__%2Fhb",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 472809 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 472809 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 472809 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 472810 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 472810 INFO  (zkCallback-706-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1'
   [junit4]   2> 472814 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 472815 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 472815 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 472825 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 472827 WARN  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 472828 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 472835 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 472839 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 472983 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 472991 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 472993 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 473002 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 473005 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 473008 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 473009 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 473010 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 473010 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 473011 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 473011 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data/]
   [junit4]   2> 473011 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2011ad98
   [junit4]   2> 473011 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data
   [junit4]   2> 473011 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data/index/
   [junit4]   2> 473011 WARN  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 473011 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data/index
   [junit4]   2> 473012 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=50, maxMergedSegmentMB=88.07421875, floorSegmentMB=0.9521484375, forceMergeDeletesPctAllowed=29.137806083283635, segmentsPerTier=27.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 473012 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@67a5708c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@be99c73; maxCacheMB=0.43359375 maxMergeSizeMB=0.8662109375)),segFN=segments_1,generation=1}
   [junit4]   2> 473012 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 473013 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data/
   [junit4]   2> 473013 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 473015 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 473015 WARN  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 473016 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 473017 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 473018 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 473018 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 473020 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 473020 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 473020 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 473020 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 473021 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 473021 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 473021 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@5c3e6152
   [junit4]   2> 473021 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@67a5708c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@be99c73; maxCacheMB=0.43359375 maxMergeSizeMB=0.8662109375)),segFN=segments_1,generation=1}
   [junit4]   2> 473021 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 473022 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@64e8b701[collection1] main]
   [junit4]   2> 473022 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 473022 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 473022 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 473022 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 473023 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 473023 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 473023 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 473023 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 473023 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 473023 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 473023 INFO  (searcherExecutor-2012-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@64e8b701[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 473024 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 473024 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 473024 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 473024 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1541120106569924608
   [junit4]   2> 473025 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1541120106569924608
   [junit4]   2> 473025 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 473026 INFO  (coreLoadExecutor-2011-thread-1-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 473026 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:45193/_/hb collection:control_collection shard:shard1
   [junit4]   2> 473027 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 473029 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96320006612516868-core_node1-n_0000000000
   [junit4]   2> 473029 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 473030 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 473031 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 473031 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45193/_/hb/collection1/
   [junit4]   2> 473031 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 473031 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 473031 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:45193/_/hb/collection1/ has no replicas
   [junit4]   2> 473031 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 473033 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96320006612516868-core_node1-n_0000000000
   [junit4]   2> 473034 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45193/_/hb/collection1/ shard1
   [junit4]   2> 473035 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:45193/_/hb",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 473136 INFO  (zkCallback-706-thread-2-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 473185 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:45193/_/hb/collection1/ and leader is http://127.0.0.1:45193/_/hb/collection1/
   [junit4]   2> 473185 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:45193/_/hb
   [junit4]   2> 473185 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 473185 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 473185 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 473186 INFO  (coreZkRegister-2005-thread-1-processing-n:127.0.0.1:45193__%2Fhb x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:45193__%2Fhb c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 473186 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45193/_/hb",
   [junit4]   2>   "node_name":"127.0.0.1:45193__%2Fhb",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 473186 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:45193/_/hb",
   [junit4]   2>   "node_name":"127.0.0.1:45193__%2Fhb",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 473289 INFO  (zkCallback-706-thread-2-processing-n:127.0.0.1:45193__%2Fhb) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 473311 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 473312 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 473313 INFO  (zkCallback-710-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7950a679 name:ZooKeeperConnection Watcher:127.0.0.1:42388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 473313 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 473313 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 473313 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 473313 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 473314 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 473314 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 473315 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 473315 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 473322 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 473322 INFO  (zkCallback-711-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3b082b74 name:ZooKeeperConnection Watcher:127.0.0.1:42388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 473322 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 473322 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 473323 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 473323 INFO  (OverseerStateUpdate-96320006612516868-127.0.0.1:45193__%2Fhb-n_0000000000) [n:127.0.0.1:45193__%2Fhb    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 473402 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/shard-1-001/cores/collection1
   [junit4]   2> 473403 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/shard-1-001
   [junit4]   2> 473403 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 473406 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@484e9764{/_/hb,null,AVAILABLE}
   [junit4]   2> 473406 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1a0c83a6{HTTP/1.1,[http/1.1]}{127.0.0.1:51260}
   [junit4]   2> 473406 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.Server Started @475689ms
   [junit4]   2> 473406 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/_/hb, hostPort=59676, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/shard-1-001/cores}
   [junit4]   2> 473407 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 473407 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/shard-1-001'
   [junit4]   2> 473407 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 473407 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 473407 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 473408 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 473408 INFO  (zkCallback-712-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1bb72e0d name:ZooKeeperConnection Watcher:127.0.0.1:42388/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 473408 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 473409 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 473409 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 473409 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/shard-1-001/solr.xml
   [junit4]   2> 473413 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.cloud.ForceLeaderTest_EA7083120FCEF20D-001/shard-1-001/cores
   [junit4]   2> 473413 INFO  (TEST-ForceLeaderTest.testLastPublishedStateIsActive-seed#[EA7083120FCEF20D]) [    ] o.a.s.c.CoreContainer New CoreContainer 853855013
   [jun

[...truncated too long message...]

ecutor-5173-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 1949104 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 1949104 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_EA7083120FCEF20D-001/solr-instance-030/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_EA7083120FCEF20D-001/solr-instance-030/./collection1/data;done=false>>]
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_EA7083120FCEF20D-001/solr-instance-030/./collection1/data
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_EA7083120FCEF20D-001/solr-instance-030/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_EA7083120FCEF20D-001/solr-instance-030/./collection1/data/index;done=false>>]
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 1949105 INFO  (coreCloseExecutor-5173-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.handler.TestReplicationHandler_EA7083120FCEF20D-001/solr-instance-030/./collection1/data/index
   [junit4]   2> 1949106 INFO  (TEST-TestReplicationHandler.doTestStopPoll-seed#[EA7083120FCEF20D]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@370cd1b8{/solr,null,UNAVAILABLE}
   [junit4]   2> 1949107 INFO  (SUITE-TestReplicationHandler-seed#[EA7083120FCEF20D]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:367)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:388)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:278)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:367)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:388)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:278)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException
   [junit4]   2> 	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
   [junit4]   2> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:347)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:367)
   [junit4]   2> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:251)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:388)
   [junit4]   2> 	at org.apache.solr.handler.ReplicationHandler.lambda$handleRequestBody$0(ReplicationHandler.java:278)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=2, maxDocsPerChunk=6, blockSize=5), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=2, blockSize=5)), sim=RandomSimilarity(queryNorm=true,coord=no): {}, locale=es-DO, timezone=America/Paramaribo
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=1,free=333089344,total=519045120
   [junit4]   2> NOTE: All tests run in this JVM: [CdcrRequestHandlerTest, RecoveryAfterSoftCommitTest, TestCollationField, CollectionStateFormat2Test, TestSolrQueryParserDefaultOperatorResource, TestIndexSearcher, TestIndexingPerformance, LeaderElectionTest, CoreSorterTest, DirectUpdateHandlerTest, FullSolrCloudDistribCmdsTest, TestOnReconnectListenerSupport, StatelessScriptUpdateProcessorFactoryTest, FullHLLTest, ConjunctionSolrSpellCheckerTest, TestDistributedMissingSort, HdfsChaosMonkeySafeLeaderTest, TestReload, TestGroupingSearch, TestSolrDeletionPolicy1, TestJmxMonitoredMap, InfoHandlerTest, TestSort, TestSuggestSpellingConverter, TestInfoStreamLogging, DistanceUnitsTest, TestNumericTerms64, TestSizeLimitedDistributedMap, BigEndianAscendingWordSerializerTest, TestBlobHandler, TestSchemaSimilarityResource, SampleTest, TestLocalFSCloudBackupRestore, TestFieldSortValues, TestManagedResource, DirectUpdateHandlerOptimizeTest, PKIAuthenticationIntegrationTest, TestDefaultStatsCache, TestPostingsSolrHighlighter, PrimitiveFieldTypeTest, UnloadDistributedZkTest, CoreAdminCreateDiscoverTest, JavabinLoaderTest, OverseerTest, FieldMutatingUpdateProcessorTest, DistributedSuggestComponentTest, TestPHPSerializedResponseWriter, TestCloudSchemaless, TestMiniSolrCloudClusterBase, TestReRankQParserPlugin, TestElisionMultitermQuery, TestHdfsCloudBackupRestore, HdfsUnloadDistributedZkTest, TestReversedWildcardFilterFactory, TestZkChroot, BinaryUpdateRequestHandlerTest, TestExactStatsCache, SliceStateTest, DistributedFacetPivotWhiteBoxTest, TestSolrQueryParser, TestSolrCLIRunExample, TestPerFieldSimilarityClassic, TestFieldCacheWithThreads, HighlighterConfigTest, FieldAnalysisRequestHandlerTest, LukeRequestHandlerTest, TestDefaultSearchFieldResource, TestQueryWrapperFilter, TestSolrJ, TestSearchPerf, TestGraphMLResponseWriter, BadComponentTest, SmileWriterTest, TestFieldCollectionResource, DocValuesMissingTest, TestUseDocValuesAsStored2, TestReplicationHandlerBackup, TestSolrIndexConfig, CircularListTest, SuggestComponentContextFilterQueryTest, RegexBoostProcessorTest, DistributedDebugComponentTest, TermVectorComponentDistributedTest, ClusterStateTest, SimpleFacetsTest, TestClusterStateMutator, TestMissingGroups, TestSolr4Spatial2, SuggesterTest, TestCollapseQParserPlugin, WrapperMergePolicyFactoryTest, TestDistribDocBasedVersion, AutoCommitTest, TestOrdValues, TestTolerantSearch, OverseerRolesTest, ConnectionManagerTest, RecoveryZkTest, TestManagedSchemaAPI, SolrTestCaseJ4Test, UpdateParamsTest, RequestHandlersTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, ShardRoutingTest, BasicZkTest, TestReplicationHandler]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandler -Dtests.seed=EA7083120FCEF20D -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=es-DO -Dtests.timezone=America/Paramaribo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J2 | TestReplicationHandler (suite) <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: ObjectTracker found 3 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EA7083120FCEF20D]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4] Completed [312/624 (4!)] on J2 in 584.61s, 15 tests, 1 failure, 1 skipped <<< FAILURES!

[...truncated 61085 lines...]



[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 143 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/143/

5 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:37465/liqc: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: KeeperErrorCode = Session expired for /overseer/queue/qn-

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37465/liqc: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: KeeperErrorCode = Session expired for /overseer/queue/qn-
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:592)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest:     1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest: 
   1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([681A53DF36E3A80E]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([681A53DF36E3A80E]:0)


FAILED:  org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest.test

Error Message:
There are still nodes recoverying - waited for 120 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 120 seconds
	at __randomizedtesting.SeedInfo.seed([681A53DF36E3A80E:E04E6C05981FC5F6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:182)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:862)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1418)
	at org.apache.solr.cloud.RecoveryZkTest.test(RecoveryZkTest.java:105)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 5 object(s) that were not released!!! [MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 5 object(s) that were not released!!! [MockDirectoryWrapper, MockDirectoryWrapper, SolrCore, MockDirectoryWrapper, MockDirectoryWrapper]
	at __randomizedtesting.SeedInfo.seed([681A53DF36E3A80E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 12094 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/init-core-data-001
   [junit4]   2> 2393737 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2393740 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 2393798 WARN  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 2393815 WARN  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2393817 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2393835 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_55224_hdfs____6xlj3y/webapp
   [junit4]   2> 2394278 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:55224
   [junit4]   2> 2394339 WARN  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2394341 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2394356 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_42371_datanode____7o4cc6/webapp
   [junit4]   2> 2394711 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42371
   [junit4]   2> 2394769 WARN  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 2394771 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 2394788 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_34818_datanode____.wsrum3/webapp
   [junit4]   2> 2394846 INFO  (IPC Server handler 4 on 38659) [    ] BlockStateChange BLOCK* processReport: from storage DS-5ab92a30-f4bd-46ed-a163-e21db4ab0559 node DatanodeRegistration(127.0.0.1:50919, datanodeUuid=229f1aad-0e95-4d56-b598-f4938e109f2e, infoPort=54636, infoSecurePort=0, ipcPort=60811, storageInfo=lv=-56;cid=testClusterID;nsid=1897704392;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 2394846 INFO  (IPC Server handler 4 on 38659) [    ] BlockStateChange BLOCK* processReport: from storage DS-6bc5a271-35ee-4572-9e48-7de81c5a14c6 node DatanodeRegistration(127.0.0.1:50919, datanodeUuid=229f1aad-0e95-4d56-b598-f4938e109f2e, infoPort=54636, infoSecurePort=0, ipcPort=60811, storageInfo=lv=-56;cid=testClusterID;nsid=1897704392;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 2395193 INFO  (SUITE-HdfsRecoveryZkTest-seed#[681A53DF36E3A80E]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34818
   [junit4]   2> 2395367 INFO  (IPC Server handler 4 on 38659) [    ] BlockStateChange BLOCK* processReport: from storage DS-43c33610-91cd-42b3-84fd-70e99edf2410 node DatanodeRegistration(127.0.0.1:41210, datanodeUuid=1029fe4a-1c32-408d-8622-b8d4edc93027, infoPort=43439, infoSecurePort=0, ipcPort=32936, storageInfo=lv=-56;cid=testClusterID;nsid=1897704392;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 2395367 INFO  (IPC Server handler 4 on 38659) [    ] BlockStateChange BLOCK* processReport: from storage DS-083325db-aa7d-4d9e-b8ad-be85d3602f2d node DatanodeRegistration(127.0.0.1:41210, datanodeUuid=1029fe4a-1c32-408d-8622-b8d4edc93027, infoPort=43439, infoSecurePort=0, ipcPort=32936, storageInfo=lv=-56;cid=testClusterID;nsid=1897704392;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 2395491 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2395492 INFO  (Thread-6585) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2395492 INFO  (Thread-6585) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2395592 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.ZkTestServer start zk server on port:41385
   [junit4]   2> 2395592 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2395594 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395596 INFO  (zkCallback-1758-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4ff52e0f name:ZooKeeperConnection Watcher:127.0.0.1:41385 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395596 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395596 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2395596 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2395598 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2395606 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395606 INFO  (zkCallback-1759-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@68402d3a name:ZooKeeperConnection Watcher:127.0.0.1:41385/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395606 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395606 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2395606 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2395608 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2395609 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2395610 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2395611 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2395611 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2395613 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 2395613 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2395614 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2395614 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2395615 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2395615 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2395617 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2395617 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2395618 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2395618 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2395619 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2395619 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2395620 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2395620 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2395622 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2395622 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2395623 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2395623 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2395624 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2395625 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2395720 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores/collection1
   [junit4]   2> 2395722 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2395746 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@480c3a48{/,null,AVAILABLE}
   [junit4]   2> 2395746 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@cbb8ac8{HTTP/1.1,[http/1.1]}{127.0.0.1:33221}
   [junit4]   2> 2395746 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.Server Started @2399044ms
   [junit4]   2> 2395746 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:38659/hdfs__localhost_38659__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001_tempDir-002_control_data, hostContext=/, hostPort=33221, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores}
   [junit4]   2> 2395746 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2395747 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001'
   [junit4]   2> 2395747 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2395747 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2395747 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2395753 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395757 INFO  (zkCallback-1760-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@617591a9 name:ZooKeeperConnection Watcher:127.0.0.1:41385/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395757 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395757 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2395761 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2395761 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/solr.xml
   [junit4]   2> 2395766 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores
   [junit4]   2> 2395766 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CoreContainer New CoreContainer 2044343108
   [junit4]   2> 2395766 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001]
   [junit4]   2> 2395766 WARN  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/lib
   [junit4]   2> 2395766 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2395767 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2395768 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2395768 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2395768 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41385/solr
   [junit4]   2> 2395768 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2395768 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2395769 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395769 INFO  (zkCallback-1763-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4e179d34 name:ZooKeeperConnection Watcher:127.0.0.1:41385 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395770 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395770 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2395773 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$10/96998034@2bd5aa6d
   [junit4]   2> 2395778 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2395778 INFO  (zkCallback-1764-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7421cdbe name:ZooKeeperConnection Watcher:127.0.0.1:41385/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2395778 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2395779 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2395780 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2395782 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2395783 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2395785 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2395786 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2395787 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2395788 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2395788 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2395792 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2395792 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 2395793 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2395794 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2395795 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2395796 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2395796 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96398334099259396-127.0.0.1:33221_-n_0000000000
   [junit4]   2> 2395796 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33221_
   [junit4]   2> 2395796 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2395797 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer Overseer (id=96398334099259396-127.0.0.1:33221_-n_0000000000) starting
   [junit4]   2> 2395798 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2395802 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2395805 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2395806 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2395809 INFO  (OverseerCollectionConfigSetProcessor-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2395812 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33221_
   [junit4]   2> 2395812 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33221_
   [junit4]   2> 2395813 INFO  (zkCallback-1764-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 2395817 INFO  (zkCallback-1764-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2395831 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2395831 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2395833 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores
   [junit4]   2> 2395834 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2395834 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores/collection1
   [junit4]   2> 2395834 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2395846 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2395846 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2395847 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 2395848 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 2395856 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 2395856 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [n:127.0.0.1:33221_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2395856 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33221",
   [junit4]   2>   "node_name":"127.0.0.1:33221_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2395856 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33221",
   [junit4]   2>   "node_name":"127.0.0.1:33221_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2395857 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2395857 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2395858 INFO  (zkCallback-1764-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2395875 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2395875 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2395875 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2395875 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2395875 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2395876 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2395876 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores/collection1'
   [junit4]   2> 2395876 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2395876 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2395877 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2395890 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2395893 WARN  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2395894 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2395903 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2395908 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2396039 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2396040 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2396040 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2396043 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2396077 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 2396077 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2396077 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 2396078 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:38659/solr_hdfs_home
   [junit4]   2> 2396078 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 2396078 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/control-001/cores/collection1], dataDir=[hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 2396078 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@259a0e9d
   [junit4]   2> 2396078 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 2396089 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 2396089 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 2396089 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 2396106 WARN  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 2401111 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 2401111 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 2401111 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 2401112 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 2401121 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 2401123 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 2401141 WARN  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:38659/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2401141 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 2401154 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 2401155 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [16777216] will allocate [1] slabs and use ~[16777216] bytes
   [junit4]   2> 2401155 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 2401186 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 2401186 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 2401186 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=12, maxMergedSegmentMB=36.7431640625, floorSegmentMB=1.541015625, forceMergeDeletesPctAllowed=6.960733237335141, segmentsPerTier=42.0, maxCFSSegmentSizeMB=1.6484375, noCFSRatio=0.26891872459162586
   [junit4]   2> 2401271 INFO  (IPC Server handler 6 on 38659) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:41210 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-6bc5a271-35ee-4572-9e48-7de81c5a14c6:NORMAL:127.0.0.1:50919|RBW], ReplicaUC[[DISK]DS-43c33610-91cd-42b3-84fd-70e99edf2410:NORMAL:127.0.0.1:41210|FINALIZED]]} size 0
   [junit4]   2> 2401273 INFO  (IPC Server handler 7 on 38659) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:50919 is added to blk_1073741825_1001 size 71
   [junit4]   2> 2401274 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7b387472); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 2401274 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2401280 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 2401280 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2401280 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2401280 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2401280 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2401280 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2401280 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2401281 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2401281 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2401281 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2401281 WARN  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2401282 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2401283 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2401284 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2401285 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2401288 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 2401288 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2401290 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 2401290 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2401290 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 2401317 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2401317 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2401319 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=76782694, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 2401343 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7b387472); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 2401343 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2401343 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7bd1df9[collection1] main]
   [junit4]   2> 2401343 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2401344 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2401344 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2401344 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2401344 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2401344 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2401345 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2401345 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2401345 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2401345 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2401346 INFO  (searcherExecutor-3732-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7bd1df9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2401346 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2401346 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2401346 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2401346 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1542373351725465600
   [junit4]   2> 2401348 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1542373351725465600
   [junit4]   2> 2401348 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2401348 INFO  (coreLoadExecutor-3731-thread-1-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2401349 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:33221 collection:control_collection shard:shard1
   [junit4]   2> 2401350 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2401353 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96398334099259396-core_node1-n_0000000000
   [junit4]   2> 2401353 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 2401355 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2401355 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2401355 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33221/collection1/
   [junit4]   2> 2401355 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2401355 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33221/collection1/ has no replicas
   [junit4]   2> 2401355 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2401356 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2401358 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96398334099259396-core_node1-n_0000000000
   [junit4]   2> 2401359 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2401359 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33221/collection1/ shard1
   [junit4]   2> 2401377 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:33221",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 2401385 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2401389 INFO  (zkCallback-1764-thread-4-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2401404 INFO  (zkCallback-1768-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59e9028a name:ZooKeeperConnection Watcher:127.0.0.1:41385/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2401412 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2401412 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2401412 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2401414 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 2401414 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2401415 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2401479 INFO  (zkCallback-1768-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2401480 INFO  (zkCallback-1764-thread-4-processing-n:127.0.0.1:33221_) [n:127.0.0.1:33221_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2401512 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:33221/collection1/ and leader is http://127.0.0.1:33221/collection1/
   [junit4]   2> 2401512 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33221
   [junit4]   2> 2401512 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2401512 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2401512 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2401513 INFO  (coreZkRegister-3724-thread-1-processing-n:127.0.0.1:33221_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33221_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 2401514 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33221",
   [junit4]   2>   "node_name":"127.0.0.1:33221_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/tlog"} current state version: 3
   [junit4]   2> 2401514 INFO  (OverseerStateUpdate-96398334099259396-127.0.0.1:33221_-n_0000000000) [n:127.0.0.1:33221_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33221",
   [junit4]   2>   "node_name":"127.0.0.1:33221_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:38659/solr_hdfs_home/control_collection/core_node1/data/tlog"}
   [junit4]   2> 2401519 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/shard-1-001/cores/collection1
   [junit4]   2> 2401520 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/shard-1-001
   [junit4]   2> 2401520 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2401522 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4151ca52{/,null,AVAILABLE}
   [junit4]   2> 2401522 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@74f34a24{HTTP/1.1,[http/1.1]}{127.0.0.1:50634}
   [junit4]   2> 2401522 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.e.j.s.Server Started @2404820ms
   [junit4]   2> 2401522 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:38659/hdfs__localhost_38659__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001_tempDir-002_jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=50634, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/shard-1-001/cores}
   [junit4]   2> 2401522 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2401522 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/shard-1-001'
   [junit4]   2> 2401523 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2401523 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2401523 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2401524 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2401524 INFO  (zkCallback-1769-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3758f651 name:ZooKeeperConnection Watcher:127.0.0.1:41385/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2401524 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2401524 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2401525 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2401525 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/shard-1-001/solr.xml
   [junit4]   2> 2401530 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsRecoveryZkTest_681A53DF36E3A80E-001/shard-1-001/cores
   [junit4]   2> 2401531 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CoreContainer New CoreContainer 1839996372
   [junit4]   2> 2401531 INFO  (TEST-HdfsRecoveryZkTest.test-seed#[681A53DF36E3A80E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jen

[...truncated too long message...]

dPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Aug 11, 2016 2:08:08 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]   2> Aug 11, 2016 2:08:11 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62): {other_tl1=PostingsFormat(name=LuceneVarGapFixedInterval), range_facet_l_dv=Lucene50(blocksize=128), _version_=PostingsFormat(name=LuceneVarGapFixedInterval), multiDefault=PostingsFormat(name=Memory doPackFST= false), a_t=PostingsFormat(name=LuceneVarGapFixedInterval), intDefault=PostingsFormat(name=LuceneVarGapFixedInterval), a_i1=PostingsFormat(name=LuceneVarGapFixedInterval), id=Lucene50(blocksize=128), range_facet_i_dv=PostingsFormat(name=LuceneVarGapFixedInterval), text=PostingsFormat(name=Asserting), range_facet_l=PostingsFormat(name=LuceneVarGapFixedInterval), timestamp=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Asserting), range_facet_i_dv=DocValuesFormat(name=Lucene54), timestamp=DocValuesFormat(name=Lucene54)}, maxPointsInLeafNode=1068, maxMBSortInHeap=5.338083296791959, sim=ClassicSimilarity, locale=cs, timezone=Antarctica/Casey
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=3,free=87685600,total=467140608
   [junit4]   2> NOTE: All tests run in this JVM: [TestHighlightDedupGrouping, OverseerTaskQueueTest, TestSolr4Spatial, TestNumericTerms32, CacheHeaderTest, BasicZkTest, FileBasedSpellCheckerTest, TestComponentsName, TestRandomFlRTGCloud, TestStressLucene, TestConfigOverlay, SimpleMLTQParserTest, TestLeaderElectionZkExpiry, CdcrRequestHandlerTest, SolrRequestParserTest, TestCloudInspectUtil, TestSolrDynamicMBean, TestLockTree, DocExpirationUpdateProcessorFactoryTest, TestLMDirichletSimilarityFactory, DeleteInactiveReplicaTest, RAMDirectoryFactoryTest, DisMaxRequestHandlerTest, TestCustomSort, TestDocSet, TestDownShardTolerantSearch, TestConfig, EchoParamsTest, ResourceLoaderTest, StandardRequestHandlerTest, PreAnalyzedFieldTest, BufferStoreTest, TlogReplayBufferedWhileIndexingTest, DeleteReplicaTest, SpatialHeatmapFacetsTest, OverseerModifyCollectionTest, JsonLoaderTest, TestRandomFaceting, TestValueSourceCache, UUIDUpdateProcessorFallbackTest, PolyFieldTest, DistributedQueueTest, HdfsChaosMonkeySafeLeaderTest, ActionThrottleTest, TestReloadDeadlock, TestHighFrequencyDictionaryFactory, TestLazyCores, QueryEqualityTest, TestManagedSchema, TestGeoJSONResponseWriter, DistributedSpellCheckComponentTest, PreAnalyzedUpdateProcessorTest, TestRestoreCore, SyncSliceTest, DistributedQueryElevationComponentTest, PathHierarchyTokenizerFactoryTest, LeaderInitiatedRecoveryOnShardRestartTest, TestUniqueKeyFieldResource, BJQParserTest, TestChildDocTransformer, TestPseudoReturnFields, JSONWriterTest, URLClassifyProcessorTest, TestMinMaxOnMultiValuedField, TestFieldCacheVsDocValues, TestNonDefinedSimilarityFactory, ExternalFileFieldSortTest, BasicDistributedZkTest, TestExactSharedStatsCache, OutputWriterTest, SolrXmlInZkTest, BigEndianAscendingWordDeserializerTest, CursorPagingTest, CleanupOldIndexTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestCrossCoreJoin, TestBlendedInfixSuggestions, SegmentsInfoRequestHandlerTest, TestPKIAuthenticationPlugin, TestReloadAndDeleteDocs, BasicAuthIntegrationTest, SynonymTokenizerTest, TestCollectionAPI, TestSmileRequest, ReplaceNodeTest, CopyFieldTest, SignatureUpdateProcessorFactoryTest, DistribCursorPagingTest, HttpPartitionTest, TestArbitraryIndexDir, HdfsDirectoryFactoryTest, TestSubQueryTransformerDistrib, TestFieldTypeCollectionResource, QueryElevationComponentTest, TestIntervalFaceting, TestCopyFieldCollectionResource, BlockJoinFacetSimpleTest, TestFieldCacheSanityChecker, TestDistributedSearch, ConnectionManagerTest, TestReplicationHandler, CollectionTooManyReplicasTest, LeaderFailoverAfterPartitionTest, OverseerCollectionConfigSetProcessorTest, OverseerStatusTest, RemoteQueryErrorTest, RollingRestartTest, SharedFSAutoReplicaFailoverUtilsTest, SimpleCollectionCreateDeleteTest, SolrCloudExampleTest, TestConfigSetsAPI, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestDistribDocBasedVersion, TestMiniSolrCloudClusterBase, TestMiniSolrCloudClusterSSL, TestOnReconnectListenerSupport, TestRebalanceLeaders, TestRequestForwarding, TestSSLRandomization, TestShortCircuitedRequests, TestSolrCloudWithDelegationTokens, TestStressCloudBlindAtomicUpdates, HdfsCollectionsAPIDistributedZkTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HdfsCollectionsAPIDistributedZkTest -Dtests.seed=681A53DF36E3A80E -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=cs -Dtests.timezone=Antarctica/Casey -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J0 | HdfsCollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest: 
   [junit4]    >    1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([681A53DF36E3A80E]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=90374, name=searcherExecutor-9610-thread-1, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([681A53DF36E3A80E]:0)
   [junit4] Completed [501/629 (3!)] on J0 in 654.30s, 1 test, 3 errors <<< FAILURES!

[...truncated 378 lines...]
   [junit4] JVM J0: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160811_124407_149.sysout
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid30416.hprof ...
   [junit4] Heap dump file created [617214879 bytes in 12.413 secs]
   [junit4] <<< JVM J0: EOF ----

[...truncated 11055 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:715: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid30416.hprof

Total time: 272 minutes 20 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 142 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/142/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=134092, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=134092, name=collection1, state=RUNNABLE, group=TGRP-CollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:34073: collection already exists: awholynewstresscollection_collection1_3
	at __randomizedtesting.SeedInfo.seed([A64F024E870E254C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:592)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)




Build Log:
[...truncated 11969 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/init-core-data-001
   [junit4]   2> 2249274 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[A64F024E870E254C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 2249275 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[A64F024E870E254C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2249290 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2249293 INFO  (Thread-12227) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2249293 INFO  (Thread-12227) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2249393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ZkTestServer start zk server on port:37694
   [junit4]   2> 2249393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2249393 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2249395 INFO  (zkCallback-32724-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@58994e50 name:ZooKeeperConnection Watcher:127.0.0.1:37694 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2249395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2249395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2249395 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2249397 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2249397 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn caught end of stream exception
   [junit4]   2> EndOfStreamException: Unable to read additional data from client sessionid 0x15674ff350f0000, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 2249400 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2249401 INFO  (zkCallback-32725-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@796e4a4 name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2249401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2249401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2249401 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2249403 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2249404 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2249405 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2249406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2249406 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2249426 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2249426 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2249427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2249427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2249428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2249428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2249429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2249430 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2249434 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2249435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2249435 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2249436 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2249437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2249437 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2249438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2249438 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2249439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2249439 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2249440 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2249440 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2249441 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2249461 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2249473 INFO  (zkCallback-32727-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1a1c7b57 name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2249474 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2249474 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2249613 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1
   [junit4]   2> 2249615 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2249722 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@48d51d6f{/,null,AVAILABLE}
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2689aa7{SSL,[ssl, http/1.1]}{127.0.0.1:34073}
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.Server Started @2252411ms
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/tempDir-001/control/data, hostContext=/, hostPort=34073, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores}
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001'
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2249724 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2249725 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2249743 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2249773 INFO  (zkCallback-32728-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@3e97244a name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2249773 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2249774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2249774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2249774 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/solr.xml
   [junit4]   2> 2249779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores
   [junit4]   2> 2249779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CoreContainer New CoreContainer 916234640
   [junit4]   2> 2249779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001]
   [junit4]   2> 2249779 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/lib
   [junit4]   2> 2249779 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2249780 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2249781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2249781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2249781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37694/solr
   [junit4]   2> 2249781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2249781 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2249794 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2249809 INFO  (zkCallback-32731-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@310de681 name:ZooKeeperConnection Watcher:127.0.0.1:37694 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2249810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2249810 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2249811 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$12/2101902147@32b69bee
   [junit4]   2> 2249833 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2249849 INFO  (zkCallback-32732-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7180f7c6 name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2249850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2249850 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2249852 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2249853 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2249855 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2249856 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2249857 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2249858 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2249858 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2249859 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2249860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2249860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 2249860 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2249862 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2249863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2249863 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2249864 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96393081490702341-127.0.0.1:34073_-n_0000000000
   [junit4]   2> 2249864 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34073_
   [junit4]   2> 2249864 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2249865 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer Overseer (id=96393081490702341-127.0.0.1:34073_-n_0000000000) starting
   [junit4]   2> 2249866 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2249869 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2249881 INFO  (OverseerCollectionConfigSetProcessor-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2249881 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2249895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34073_
   [junit4]   2> 2249895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:34073_
   [junit4]   2> 2249895 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2249896 INFO  (zkCallback-32732-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 2249897 INFO  (zkCallback-32732-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2249909 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2249910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2249910 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores
   [junit4]   2> 2249911 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2249911 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1
   [junit4]   2> 2249911 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2249912 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2249912 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2249912 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 2249912 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:34073_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2249913 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 2249913 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:34073",
   [junit4]   2>   "node_name":"127.0.0.1:34073_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2249914 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:34073",
   [junit4]   2>   "node_name":"127.0.0.1:34073_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2249914 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2249914 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 2249914 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2249915 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2249915 INFO  (zkCallback-32732-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2250916 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2250916 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2250916 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2250916 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2250917 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2250917 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1'
   [junit4]   2> 2250917 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2250917 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2250918 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2250922 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2250925 WARN  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2250926 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2250980 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2250985 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2251087 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2251093 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2251094 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2251101 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2251103 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2251104 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2251104 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2251106 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 2251106 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2251106 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2251106 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/]
   [junit4]   2> 2251106 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@2c21dbd4
   [junit4]   2> 2251107 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/snapshot_metadata
   [junit4]   2> 2251107 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 2251107 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data
   [junit4]   2> 2251107 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2251107 WARN  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2251107 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/index
   [junit4]   2> 2251107 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=16, maxMergeAtOnceExplicit=38, maxMergedSegmentMB=0.7978515625, floorSegmentMB=0.5205078125, forceMergeDeletesPctAllowed=17.571953688359336, segmentsPerTier=14.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8681645510787229
   [junit4]   2> 2251108 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@15174f87 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7fbb82bb),segFN=segments_1,generation=1}
   [junit4]   2> 2251108 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2251141 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2251141 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2251142 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2251142 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2251142 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2251142 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2251149 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2251150 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2251150 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2251150 WARN  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2251152 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2251152 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2251153 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2251154 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2251156 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 2251156 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2251157 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/
   [junit4]   2> 2251157 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2251167 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2251167 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2251168 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2251168 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2251168 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=10, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2251169 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@15174f87 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7fbb82bb),segFN=segments_1,generation=1}
   [junit4]   2> 2251169 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2251169 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@48016e23[collection1] main]
   [junit4]   2> 2251169 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2251174 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2251174 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2251174 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2251174 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2251174 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2251175 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2251175 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2251175 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2251175 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2251176 INFO  (searcherExecutor-10491-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@48016e23[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2251176 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2251176 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2251176 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2251176 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1542289305824657408
   [junit4]   2> 2251178 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1542289305824657408
   [junit4]   2> 2251178 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2251178 INFO  (coreLoadExecutor-10490-thread-1-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2251179 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:34073 collection:control_collection shard:shard1
   [junit4]   2> 2251180 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2251182 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96393081490702341-core_node1-n_0000000000
   [junit4]   2> 2251182 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 2251184 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2251184 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2251184 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:34073/collection1/
   [junit4]   2> 2251184 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2251184 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:34073/collection1/ has no replicas
   [junit4]   2> 2251184 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2251185 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2251187 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96393081490702341-core_node1-n_0000000000
   [junit4]   2> 2251189 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:34073/collection1/ shard1
   [junit4]   2> 2251190 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:34073",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2251292 INFO  (zkCallback-32732-thread-2-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2251339 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:34073/collection1/ and leader is https://127.0.0.1:34073/collection1/
   [junit4]   2> 2251340 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:34073
   [junit4]   2> 2251340 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2251340 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2251340 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2251341 INFO  (coreZkRegister-10484-thread-1-processing-n:127.0.0.1:34073_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:34073_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 2251341 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:34073",
   [junit4]   2>   "node_name":"127.0.0.1:34073_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 2251341 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:34073",
   [junit4]   2>   "node_name":"127.0.0.1:34073_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2251414 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2251419 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2251427 INFO  (zkCallback-32736-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2de06f22 name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2251427 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2251428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2251428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2251428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 2251428 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2251429 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2251454 INFO  (zkCallback-32736-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2251455 INFO  (zkCallback-32732-thread-2-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2251542 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/cores/collection1
   [junit4]   2> 2251543 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001
   [junit4]   2> 2251545 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2251610 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@65d23177{/,null,AVAILABLE}
   [junit4]   2> 2251611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6e4056a6{SSL,[ssl, http/1.1]}{127.0.0.1:42054}
   [junit4]   2> 2251611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.e.j.s.Server Started @2254298ms
   [junit4]   2> 2251611 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=42054, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/cores}
   [junit4]   2> 2251612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2251612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001'
   [junit4]   2> 2251612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2251612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2251612 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2251627 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2251634 INFO  (zkCallback-32737-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5ef792bc name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2251634 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2251635 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2251636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2251636 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/solr.xml
   [junit4]   2> 2251641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/cores
   [junit4]   2> 2251641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CoreContainer New CoreContainer 495292304
   [junit4]   2> 2251641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001]
   [junit4]   2> 2251641 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/lib
   [junit4]   2> 2251641 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2251649 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2251657 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2251658 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2251658 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37694/solr
   [junit4]   2> 2251658 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2251658 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2251666 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2251677 INFO  (zkCallback-32740-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4d82fed9 name:ZooKeeperConnection Watcher:127.0.0.1:37694 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2251678 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2251678 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2251679 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$12/2101902147@60ef4c78
   [junit4]   2> 2251697 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2251702 INFO  (zkCallback-32741-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2f7c1d5a name:ZooKeeperConnection Watcher:127.0.0.1:37694/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2251703 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2251706 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2251707 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 2251707 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2251709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2251709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96393081490702345-127.0.0.1:42054_-n_0000000001
   [junit4]   2> 2251709 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96393081490702341-127.0.0.1:34073_-n_0000000000 to know if I could be the leader
   [junit4]   2> 2251710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.ZkController Publish node=127.0.0.1:42054_ as DOWN
   [junit4]   2> 2251710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42054_
   [junit4]   2> 2251710 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:42054_
   [junit4]   2> 2251711 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:42054_"} current state version: 3
   [junit4]   2> 2251711 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:42054_
   [junit4]   2> 2251711 INFO  (zkCallback-32736-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2251711 INFO  (zkCallback-32732-thread-2-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2251711 INFO  (zkCallback-32741-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2251712 INFO  (zkCallback-32736-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2251712 INFO  (zkCallback-32732-thread-2-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2251715 INFO  (zkCallback-32741-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2251751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2251751 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2251752 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/cores
   [junit4]   2> 2251753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2251753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/cores/collection1
   [junit4]   2> 2251753 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2251766 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2251766 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2251767 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController Collection collection1 not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 2251767 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:42054",
   [junit4]   2>   "node_name":"127.0.0.1:42054_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 2251767 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:42054",
   [junit4]   2>   "node_name":"127.0.0.1:42054_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"collection1",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2251767 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2251767 INFO  (OverseerStateUpdate-96393081490702341-127.0.0.1:34073_-n_0000000000) [n:127.0.0.1:34073_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 2251767 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [collection1]
   [junit4]   2> 2251788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 2251788 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[A64F024E870E254C]) [n:127.0.0.1:42054_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2251798 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2251869 INFO  (zkCallback-32736-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251869 INFO  (zkCallback-32732-thread-2-processing-n:127.0.0.1:34073_) [n:127.0.0.1:34073_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2251869 INFO  (zkCallback-32741-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2252798 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2252798 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController Check for collection zkNode:collection1
   [junit4]   2> 2252799 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2252799 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/collection1]
   [junit4]   2> 2252799 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/collection1] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2252799 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_A64F024E870E254C-001/shard-1-001/cores/collection1'
   [junit4]   2> 2252799 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2252799 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2252801 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2252805 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2252808 WARN  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2252809 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2252833 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2252839 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2253033 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2253039 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2253040 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2253049 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2253050 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2253062 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2253062 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2253064 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 2253064 INFO  (coreLoadExecutor-10501-thread-1-processing-n:127.0.0.1:42054_) [n:127.0.0.1:42054_ c:collection1   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' 

[...truncated too long message...]

s-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160810_143044_914.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160810_143044_914.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 265 minutes 49 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J2-20160810_143044_914.events=solr/build/solr-core/test/temp/junit4-J2-20160810_143044_914.events, heapdumps/java_pid25425.hprof=heapdumps/java_pid25425.hprof, solr/build/solr-core/test/temp/junit4-J0-20160810_143044_911.events=solr/build/solr-core/test/temp/junit4-J0-20160810_143044_911.events, solr/build/solr-core/test/temp/junit4-J1-20160810_143044_911.events=solr/build/solr-core/test/temp/junit4-J1-20160810_143044_911.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 141 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/141/

3 tests failed.
FAILED:  org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:46010/forceleader_test_collection_shard1_replica1]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live SolrServers available to handle this request:[http://127.0.0.1:46010/forceleader_test_collection_shard1_replica1]
	at __randomizedtesting.SeedInfo.seed([6F667FE33CB5C521:89F14B2305373C40]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:753)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.sendDocsWithRetry(AbstractFullDistribZkTestBase.java:741)
	at org.apache.solr.cloud.ForceLeaderTest.sendDoc(ForceLeaderTest.java:424)
	at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:131)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:46010/forceleader_test_collection_shard1_replica1]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:735)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more
Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:46010/forceleader_test_collection_shard1_replica1
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:599)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:372)
	... 6 more
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
	... 10 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsSyncSliceTest.test

Error Message:
There are still nodes recoverying - waited for 30 seconds

Stack Trace:
java.lang.AssertionError: There are still nodes recoverying - waited for 30 seconds
	at __randomizedtesting.SeedInfo.seed([6F667FE33CB5C521:E73240399249A8D9]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:182)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:862)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForThingsToLevelOut(AbstractFullDistribZkTestBase.java:1418)
	at org.apache.solr.cloud.SyncSliceTest.test(SyncSliceTest.java:71)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart

Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1

Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
	at __randomizedtesting.SeedInfo.seed([6F667FE33CB5C521:337B9862F2957117]:0)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:794)
	at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart(CdcrReplicationDistributedZkTest.java:264)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11474 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsSyncSliceTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/init-core-data-001
   [junit4]   2> 793576 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 793578 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /msn/my
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 793610 WARN  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 793616 WARN  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 793618 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 793625 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_61000_hdfs____.y1o7nl/webapp
   [junit4]   2> 793938 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:61000
   [junit4]   2> 794006 WARN  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 794007 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 794018 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_58608_datanode____7pu46a/webapp
   [junit4]   2> 794422 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:58608
   [junit4]   2> 794459 WARN  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 794459 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 794471 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_40843_datanode____mj7y1u/webapp
   [junit4]   2> 794546 INFO  (IPC Server handler 3 on 43159) [    ] BlockStateChange BLOCK* processReport: from storage DS-82809e76-64b9-4878-8d10-18a57e1357ef node DatanodeRegistration(127.0.0.1:34154, datanodeUuid=052daa0d-a8e9-4386-b794-88085936c789, infoPort=37636, infoSecurePort=0, ipcPort=41893, storageInfo=lv=-56;cid=testClusterID;nsid=447807271;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 794546 INFO  (IPC Server handler 3 on 43159) [    ] BlockStateChange BLOCK* processReport: from storage DS-c61bfa81-06e9-4aad-9d0c-583bece9b712 node DatanodeRegistration(127.0.0.1:34154, datanodeUuid=052daa0d-a8e9-4386-b794-88085936c789, infoPort=37636, infoSecurePort=0, ipcPort=41893, storageInfo=lv=-56;cid=testClusterID;nsid=447807271;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 794839 INFO  (SUITE-HdfsSyncSliceTest-seed#[6F667FE33CB5C521]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40843
   [junit4]   2> 794908 INFO  (IPC Server handler 8 on 43159) [    ] BlockStateChange BLOCK* processReport: from storage DS-d52833c7-fb03-4b33-acaa-06001e10c330 node DatanodeRegistration(127.0.0.1:37007, datanodeUuid=171cdc8a-684f-47d8-9345-0d5235857205, infoPort=47138, infoSecurePort=0, ipcPort=53015, storageInfo=lv=-56;cid=testClusterID;nsid=447807271;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 794909 INFO  (IPC Server handler 8 on 43159) [    ] BlockStateChange BLOCK* processReport: from storage DS-d28cdf0d-481d-4d39-b5a9-4cbd7cced32f node DatanodeRegistration(127.0.0.1:37007, datanodeUuid=171cdc8a-684f-47d8-9345-0d5235857205, infoPort=47138, infoSecurePort=0, ipcPort=53015, storageInfo=lv=-56;cid=testClusterID;nsid=447807271;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 794977 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 794977 INFO  (Thread-6000) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 794978 INFO  (Thread-6000) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 795078 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.ZkTestServer start zk server on port:59746
   [junit4]   2> 795078 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 795078 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 795080 INFO  (zkCallback-547-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@70454e0f name:ZooKeeperConnection Watcher:127.0.0.1:59746 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 795080 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 795080 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 795080 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 795082 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 795082 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 795083 INFO  (zkCallback-548-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@61baffb4 name:ZooKeeperConnection Watcher:127.0.0.1:59746/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 795083 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 795083 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 795083 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 795084 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 795085 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 795085 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 795086 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 795086 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 795087 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 795087 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 795088 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 795088 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 795089 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 795089 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 795090 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 795090 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 795091 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 795091 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 795091 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 795092 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 795092 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 795092 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 795093 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 795093 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 795094 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 795094 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 795095 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 795095 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 795096 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 795096 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 795096 INFO  (zkCallback-550-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@693a793e name:ZooKeeperConnection Watcher:127.0.0.1:59746/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 795097 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 795097 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 795157 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores/collection1
   [junit4]   2> 795158 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 795160 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@574f60da{/msn/my,null,AVAILABLE}
   [junit4]   2> 795161 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.e.j.s.ServerConnector Started ServerConnector@14b103c9{SSL,[ssl, http/1.1]}{127.0.0.1:57949}
   [junit4]   2> 795161 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.e.j.s.Server Started @797539ms
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost:43159/hdfs__localhost_43159__x1_jenkins_jenkins-slave_workspace_Lucene-Solr-NightlyTests-6.x_solr_build_solr-core_test_J1_temp_solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001_tempDir-002_control_data, hostContext=/msn/my, hostPort=57949, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores}
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001'
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 795162 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 795163 INFO  (zkCallback-551-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4c07871f name:ZooKeeperConnection Watcher:127.0.0.1:59746/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 795163 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 795163 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 795164 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 795164 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/solr.xml
   [junit4]   2> 795167 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores
   [junit4]   2> 795167 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.CoreContainer New CoreContainer 1356455593
   [junit4]   2> 795168 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001]
   [junit4]   2> 795168 WARN  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/lib
   [junit4]   2> 795168 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59746/solr
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 795169 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 795170 INFO  (zkCallback-554-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@f403764 name:ZooKeeperConnection Watcher:127.0.0.1:59746 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 795170 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 795171 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 795171 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$79/1447441381@1b911e2c
   [junit4]   2> 795171 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 795172 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2afb88a1 name:ZooKeeperConnection Watcher:127.0.0.1:59746/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 795172 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 795172 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 795173 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 795175 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 795175 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 795176 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 795177 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 795178 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 795178 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 795179 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 795179 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 795179 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 795180 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 795181 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 795181 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 795182 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 795182 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000
   [junit4]   2> 795182 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57949_msn%2Fmy
   [junit4]   2> 795183 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 795183 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer Overseer (id=96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) starting
   [junit4]   2> 795184 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 795191 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 795192 INFO  (OverseerCollectionConfigSetProcessor-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 795192 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 795192 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57949_msn%2Fmy
   [junit4]   2> 795192 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:57949_msn%2Fmy
   [junit4]   2> 795194 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 795194 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 795195 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 795202 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 795202 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 795202 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores
   [junit4]   2> 795203 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 795203 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores/collection1
   [junit4]   2> 795203 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 795203 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 795203 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 795203 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 795203 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 795204 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 795204 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:57949/msn/my",
   [junit4]   2>   "node_name":"127.0.0.1:57949_msn%2Fmy",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 795204 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:57949/msn/my",
   [junit4]   2>   "node_name":"127.0.0.1:57949_msn%2Fmy",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 795204 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 795204 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 795204 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 795205 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 795205 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 796206 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 796206 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 796206 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 796206 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 796207 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 796207 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores/collection1'
   [junit4]   2> 796207 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 796207 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 796209 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 796212 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 796215 WARN  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 796216 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 796223 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 796227 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 796304 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 796308 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 796309 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 796313 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 796314 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 796315 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 796316 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 796316 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 796316 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 796317 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 796317 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:43159/solr_hdfs_home
   [junit4]   2> 796317 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 796317 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.hdfs.HdfsSyncSliceTest_6F667FE33CB5C521-001/control-001/cores/collection1], dataDir=[hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 796317 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@732ae4db
   [junit4]   2> 796317 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 796322 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 796322 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 796322 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 796329 WARN  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 801330 WARN  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
   [junit4]   2> 806333 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 806334 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 806334 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 806334 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 806342 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 806342 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 806348 WARN  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:43159/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 806348 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 806354 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 806354 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 806354 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 806361 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 806361 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 806361 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6e4f3aeb
   [junit4]   2> 806372 INFO  (IPC Server handler 2 on 43159) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:37007 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-c61bfa81-06e9-4aad-9d0c-583bece9b712:NORMAL:127.0.0.1:34154|RBW], ReplicaUC[[DISK]DS-d52833c7-fb03-4b33-acaa-06001e10c330:NORMAL:127.0.0.1:37007|FINALIZED]]} size 0
   [junit4]   2> 806373 INFO  (IPC Server handler 1 on 43159) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34154 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-d52833c7-fb03-4b33-acaa-06001e10c330:NORMAL:127.0.0.1:37007|FINALIZED], ReplicaUC[[DISK]DS-82809e76-64b9-4878-8d10-18a57e1357ef:NORMAL:127.0.0.1:34154|FINALIZED]]} size 0
   [junit4]   2> 806374 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@944aa68); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 806374 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 806375 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 806377 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 806377 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 806377 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 806378 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 806378 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 806378 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 806378 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 806378 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 806378 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 806378 WARN  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 806379 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 806380 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 806380 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 806381 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 806383 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 806383 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 806384 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 806384 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 806384 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 806400 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 806400 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 806401 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=951729629, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 806404 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@944aa68); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 806404 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 806404 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@96f88d0[collection1] main]
   [junit4]   2> 806404 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 806405 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 806406 INFO  (searcherExecutor-1562-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@96f88d0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 806406 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 806406 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 806406 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 806406 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1542209297904041984
   [junit4]   2> 806408 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1542209297904041984
   [junit4]   2> 806408 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 806408 INFO  (coreLoadExecutor-1561-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 806409 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:57949/msn/my collection:control_collection shard:shard1
   [junit4]   2> 806409 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 806411 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96388080370057221-core_node1-n_0000000000
   [junit4]   2> 806412 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 806413 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 806413 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 806413 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 806413 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:57949/msn/my/collection1/
   [junit4]   2> 806413 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 806413 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:57949/msn/my/collection1/ has no replicas
   [junit4]   2> 806414 INFO  (zkCallback-555-thread-2-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 806414 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 806415 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96388080370057221-core_node1-n_0000000000
   [junit4]   2> 806416 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:57949/msn/my/collection1/ shard1
   [junit4]   2> 806417 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:57949/msn/my",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 806518 INFO  (zkCallback-555-thread-2-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 806567 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:57949/msn/my/collection1/ and leader is https://127.0.0.1:57949/msn/my/collection1/
   [junit4]   2> 806567 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:57949/msn/my
   [junit4]   2> 806567 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 806567 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 806568 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 806568 INFO  (coreZkRegister-1555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:57949_msn%2Fmy c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 806569 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:57949/msn/my",
   [junit4]   2>   "node_name":"127.0.0.1:57949_msn%2Fmy",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/tlog"} current state version: 3
   [junit4]   2> 806569 INFO  (OverseerStateUpdate-96388080370057221-127.0.0.1:57949_msn%2Fmy-n_0000000000) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:57949/msn/my",
   [junit4]   2>   "node_name":"127.0.0.1:57949_msn%2Fmy",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:43159/solr_hdfs_home/control_collection/core_node1/data/tlog"}
   [junit4]   2> 806670 INFO  (zkCallback-555-thread-1-processing-n:127.0.0.1:57949_msn%2Fmy) [n:127.0.0.1:57949_msn%2Fmy    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 806707 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 806708 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 806709 INFO  (zkCallback-559-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7f9fd71f name:ZooKeeperConnection Watcher:127.0.0.1:59746/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 806709 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 806709 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 806709 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 806709 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 806709 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 806710 INFO  (TEST-HdfsSyncSliceTest.test-seed#[6F667FE33CB5C521]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause

[...truncated too long message...]

exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160809_174309_109.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160809_174309_109.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 586 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 140 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/140/

2 tests failed.
FAILED:  org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart

Error Message:
Timeout waiting for CDCR replication to complete @source_collection:shard1

Stack Trace:
java.lang.RuntimeException: Timeout waiting for CDCR replication to complete @source_collection:shard1
	at __randomizedtesting.SeedInfo.seed([C519C4A7A908694E:990423266728DD78]:0)
	at org.apache.solr.cloud.BaseCdcrDistributedZkTest.waitForReplicationToComplete(BaseCdcrDistributedZkTest.java:794)
	at org.apache.solr.cloud.CdcrReplicationDistributedZkTest.testReplicationAfterRestart(CdcrReplicationDistributedZkTest.java:264)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:58224

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:58224
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:603)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:457)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
	... 47 more




Build Log:
[...truncated 11767 lines...]
   [junit4] JVM J2: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160808_110251_850.sysout
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid9110.hprof ...
   [junit4] Heap dump file created [735374596 bytes in 9.294 secs]
   [junit4] <<< JVM J2: EOF ----

   [junit4] JVM J2: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160808_110251_850.syserr
   [junit4] >>> JVM J2 emitted unexpected output (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J2: EOF ----

[...truncated 80 lines...]
   [junit4] Suite: org.apache.solr.cloud.CdcrReplicationDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/init-core-data-001
   [junit4]   2> 302829 INFO  (SUITE-CdcrReplicationDistributedZkTest-seed#[C519C4A7A908694E]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 302830 INFO  (SUITE-CdcrReplicationDistributedZkTest-seed#[C519C4A7A908694E]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 302833 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 302833 INFO  (Thread-789) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 302833 INFO  (Thread-789) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 302933 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.ZkTestServer start zk server on port:41903
   [junit4]   2> 302933 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 302934 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 302937 INFO  (zkCallback-492-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@27e010 name:ZooKeeperConnection Watcher:127.0.0.1:41903 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 302937 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 302937 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 302937 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 302941 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 302941 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 302944 INFO  (zkCallback-493-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5de3642b name:ZooKeeperConnection Watcher:127.0.0.1:41903/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 302944 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 302945 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 302945 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 302947 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 302949 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 302950 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 302952 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-cdcr.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 302952 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 302954 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 302954 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 302957 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 302957 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 302958 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 302958 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 302960 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 302960 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 302962 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 302962 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 302963 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 302963 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 302966 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 302966 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 302974 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 302974 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 302977 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 302977 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 302979 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 302979 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 302980 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 302981 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 302982 INFO  (zkCallback-495-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2b0f04f3 name:ZooKeeperConnection Watcher:127.0.0.1:41903/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 302982 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 302982 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 302984 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBufferOnNonLeader
   [junit4]   2> 303053 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1
   [junit4]   2> 303055 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 303056 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62f70b32{/,null,AVAILABLE}
   [junit4]   2> 303064 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2a76fbe7{SSL,[ssl, http/1.1]}{127.0.0.1:50701}
   [junit4]   2> 303064 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.Server Started @305217ms
   [junit4]   2> 303064 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard1, hostContext=/, hostPort=50701, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores}
   [junit4]   2> 303064 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 303065 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001'
   [junit4]   2> 303065 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 303065 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 303065 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 303065 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 303066 INFO  (zkCallback-496-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57538a16 name:ZooKeeperConnection Watcher:127.0.0.1:41903/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 303067 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 303067 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 303067 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 303068 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/solr.xml
   [junit4]   2> 303071 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores
   [junit4]   2> 303071 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CoreContainer New CoreContainer 808811889
   [junit4]   2> 303071 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001]
   [junit4]   2> 303071 WARN  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/lib
   [junit4]   2> 303071 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 303072 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 303073 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 303073 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 303073 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41903/solr
   [junit4]   2> 303073 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 303073 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 303073 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 303077 INFO  (zkCallback-499-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@587827f6 name:ZooKeeperConnection Watcher:127.0.0.1:41903 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 303077 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 303077 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 303078 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$10/276052250@950bd52
   [junit4]   2> 303078 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 303079 INFO  (zkCallback-500-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@57aaf1a8 name:ZooKeeperConnection Watcher:127.0.0.1:41903/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 303079 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 303080 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 303082 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 303083 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 303085 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 303086 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 303087 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 303087 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 303088 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 303089 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 303090 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 303090 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 303091 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 303093 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 303093 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 303095 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 303095 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96380811796807685-127.0.0.1:50701_-n_0000000000
   [junit4]   2> 303095 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:50701_
   [junit4]   2> 303096 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 303097 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer Overseer (id=96380811796807685-127.0.0.1:50701_-n_0000000000) starting
   [junit4]   2> 303098 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 303103 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 303103 INFO  (OverseerCollectionConfigSetProcessor-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 303104 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50701_
   [junit4]   2> 303104 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:50701_
   [junit4]   2> 303104 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 303105 INFO  (zkCallback-500-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 303105 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 303112 INFO  (zkCallback-500-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (1)
   [junit4]   2> 303115 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 303115 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 303115 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores
   [junit4]   2> 303116 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=tmp_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 303116 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1
   [junit4]   2> 303116 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 303116 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 303116 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 303117 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 303117 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:50701_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 303117 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Collection tmp_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 303117 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [tmp_collection]
   [junit4]   2> 303118 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:50701",
   [junit4]   2>   "node_name":"127.0.0.1:50701_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 303118 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:50701",
   [junit4]   2>   "node_name":"127.0.0.1:50701_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 303118 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.o.ClusterStateMutator building a new cName: tmp_collection
   [junit4]   2> 303118 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 303118 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
   [junit4]   2> 303119 INFO  (zkCallback-500-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 304118 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 304118 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:tmp_collection
   [junit4]   2> 304119 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Creating collection in ZooKeeper:tmp_collection
   [junit4]   2> 304119 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Looking for collection configName
   [junit4]   2> 304119 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Only one config set found in zk - using it:conf1
   [junit4]   2> 304120 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection
   [junit4]   2> 304121 INFO  (zkCallback-500-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader A collections change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections], has occurred - updating...
   [junit4]   2> 304121 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/tmp_collection]
   [junit4]   2> 304122 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/tmp_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 304122 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1'
   [junit4]   2> 304122 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 304122 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 304124 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 304132 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 304134 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 304139 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 304145 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 304212 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 304213 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 304213 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 304216 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 304243 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 304243 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection tmp_collection
   [junit4]   2> 304244 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 304244 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/]
   [junit4]   2> 304244 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1b94d521
   [junit4]   2> 304244 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/snapshot_metadata
   [junit4]   2> 304244 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 304245 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data
   [junit4]   2> 304245 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/index/
   [junit4]   2> 304245 WARN  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 304245 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/index
   [junit4]   2> 304246 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=575335063, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 304246 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@27214a89 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@63d07e4),segFN=segments_1,generation=1}
   [junit4]   2> 304246 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 304247 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/
   [junit4]   2> 304247 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 304249 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "cdcr-processor-chain"
   [junit4]   2> 304249 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 304250 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 304251 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 304251 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 304252 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 304256 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/cdcr,/update,/admin/file,/terms
   [junit4]   2> 304256 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 304256 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 304256 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 304257 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 304257 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 304257 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=14, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4610309097504345]
   [junit4]   2> 304257 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@27214a89 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@63d07e4),segFN=segments_1,generation=1}
   [junit4]   2> 304257 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 304257 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5b0b6264[collection1] main]
   [junit4]   2> 304258 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/tmp_collection]
   [junit4]   2> 304258 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/tmp_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 304258 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 304258 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 304258 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 304258 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 304259 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 304260 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 304260 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 304260 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 304260 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection/cdcr/state
   [junit4]   2> 304263 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.CdcrBufferStateManager Created znode /collections/tmp_collection/cdcr/state/buffer
   [junit4]   2> 304264 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.h.CdcrProcessStateManager Created znode /collections/tmp_collection/cdcr/state/process
   [junit4]   2> 304265 INFO  (searcherExecutor-1500-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5b0b6264[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 304265 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 304265 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 304265 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 304265 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1542092990249435136
   [junit4]   2> 304267 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 2.0ms to seed version buckets with highest version 1542092990249435136
   [junit4]   2> 304267 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 304267 INFO  (coreLoadExecutor-1499-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 304268 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:https://127.0.0.1:50701 collection:tmp_collection shard:shard2
   [junit4]   2> 304269 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection/leader_elect/shard2/election
   [junit4]   2> 304278 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/tmp_collection/leader_elect/shard2/election/96380811796807685-core_node1-n_0000000000
   [junit4]   2> 304278 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard2 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 304280 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 304280 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 304280 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:50701/collection1/
   [junit4]   2> 304280 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 304280 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:50701/collection1/ has no replicas
   [junit4]   2> 304281 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"tmp_collection"} current state version: 1
   [junit4]   2> 304281 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/tmp_collection/leaders/shard2
   [junit4]   2> 304283 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/tmp_collection/leaders/shard2/leader after winning as /collections/tmp_collection/leader_elect/shard2/election/96380811796807685-core_node1-n_0000000000
   [junit4]   2> 304283 INFO  (zkCallback-500-thread-2-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ tmp_collection:shard2
   [junit4]   2> 304284 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:50701/collection1/ shard2
   [junit4]   2> 304285 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "base_url":"https://127.0.0.1:50701",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 304386 INFO  (zkCallback-500-thread-2-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 304435 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController We are https://127.0.0.1:50701/collection1/ and leader is https://127.0.0.1:50701/collection1/
   [junit4]   2> 304435 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=https://127.0.0.1:50701
   [junit4]   2> 304436 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 304436 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 304436 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 304436 INFO  (coreZkRegister-1493-thread-1-processing-n:127.0.0.1:50701_ x:collection1 s:shard2 c:tmp_collection r:core_node1) [n:127.0.0.1:50701_ c:tmp_collection s:shard2 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection tmp_collection
   [junit4]   2> 304437 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:50701",
   [junit4]   2>   "node_name":"127.0.0.1:50701_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 304437 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:50701",
   [junit4]   2>   "node_name":"127.0.0.1:50701_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 304541 INFO  (zkCallback-500-thread-1-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 304692 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/cores/collection1
   [junit4]   2> 304694 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 304696 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e703e69{/,null,AVAILABLE}
   [junit4]   2> 304699 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2eb367ed{SSL,[ssl, http/1.1]}{127.0.0.1:47996}
   [junit4]   2> 304699 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.e.j.s.Server Started @306852ms
   [junit4]   2> 304699 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard2, hostContext=/, hostPort=47996, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/cores}
   [junit4]   2> 304700 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 304700 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002'
   [junit4]   2> 304700 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 304700 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 304702 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 304702 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 304704 INFO  (zkCallback-502-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@59ba42bb name:ZooKeeperConnection Watcher:127.0.0.1:41903/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 304704 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 304704 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 304705 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 304705 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/solr.xml
   [junit4]   2> 304709 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/cores
   [junit4]   2> 304709 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CoreContainer New CoreContainer 900869275
   [junit4]   2> 304709 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002]
   [junit4]   2> 304709 WARN  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/lib
   [junit4]   2> 304710 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 304711 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 304711 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 304711 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 304711 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41903/solr
   [junit4]   2> 304711 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 304711 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 304716 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 304718 INFO  (zkCallback-505-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@4b8152b9 name:ZooKeeperConnection Watcher:127.0.0.1:41903 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 304718 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 304718 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 304719 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$10/276052250@48471094
   [junit4]   2> 304721 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 304722 INFO  (zkCallback-506-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6b1f9a2c name:ZooKeeperConnection Watcher:127.0.0.1:41903/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 304722 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 304735 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 304736 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.c.ZkStateReader Loaded cluster properties: {urlScheme=https}
   [junit4]   2> 304736 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 304740 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 304740 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96380811796807688-127.0.0.1:47996_-n_0000000001
   [junit4]   2> 304741 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96380811796807685-127.0.0.1:50701_-n_0000000000 to know if I could be the leader
   [junit4]   2> 304741 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.ZkController Publish node=127.0.0.1:47996_ as DOWN
   [junit4]   2> 304743 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47996_
   [junit4]   2> 304743 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:47996_
   [junit4]   2> 304743 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:47996_"} current state version: 3
   [junit4]   2> 304743 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:47996_
   [junit4]   2> 304744 INFO  (zkCallback-500-thread-3-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 304744 INFO  (zkCallback-506-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 304745 INFO  (zkCallback-500-thread-3-processing-n:127.0.0.1:50701_) [n:127.0.0.1:50701_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 304749 INFO  (zkCallback-506-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 304755 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 304755 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 304756 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/cores
   [junit4]   2> 304757 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=tmp_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 304757 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrReplicationDistributedZkTest_C519C4A7A908694E-001/jetty-002/cores/collection1
   [junit4]   2> 304757 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 304758 INFO  (coreLoadExecutor-1511-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_ c:tmp_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 304758 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 304758 INFO  (coreLoadExecutor-1511-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_ c:tmp_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 304758 INFO  (TEST-CdcrReplicationDistributedZkTest.testBufferOnNonLeader-seed#[C519C4A7A908694E]) [n:127.0.0.1:47996_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 304759 INFO  (coreLoadExecutor-1511-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_ c:tmp_collection   x:collection1] o.a.s.c.ZkController Registering watch for collection tmp_collection
   [junit4]   2> 304759 INFO  (coreLoadExecutor-1511-thread-1-processing-n:127.0.0.1:47996_) [n:127.0.0.1:47996_ c:tmp_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [tmp_collection]
   [junit4]   2> 304759 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_0000000000) [n:127.0.0.1:50701_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"https://127.0.0.1:47996",
   [junit4]   2>   "node_name":"127.0.0.1:47996_",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"tmp_collection",
   [junit4]   2>   "operation":"state"} current state version: 3
   [junit4]   2> 304760 INFO  (OverseerStateUpdate-96380811796807685-127.0.0.1:50701_-n_000

[...truncated too long message...]

exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160808_110251_850.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160808_110251_850.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 295 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 139 - Failure

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/139/

3 tests failed.
FAILED:  org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test

Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:55214/hd_om/c8n_1x3_lf_shard1_replica2]

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: No live SolrServers available to handle this request:[http://127.0.0.1:55214/hd_om/c8n_1x3_lf_shard1_replica2]
	at __randomizedtesting.SeedInfo.seed([4C17D787E792729C:C443E85D496E1F64]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:760)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1172)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:592)
	at org.apache.solr.cloud.HttpPartitionTest.sendDoc(HttpPartitionTest.java:578)
	at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:174)
	at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:55)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:55214/hd_om/c8n_1x3_lf_shard1_replica2]
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:383)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:735)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more
Caused by: org.apache.solr.client.solrj.SolrServerException: Server refused connection at: http://127.0.0.1:55214/hd_om/c8n_1x3_lf_shard1_replica2
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:599)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
	... 6 more
Caused by: java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:589)
	at org.apache.http.conn.scheme.PlainSocketFactory.connectSocket(PlainSocketFactory.java:117)
	at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:177)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:304)
	at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:611)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:446)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
	... 10 more


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:48129

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:48129
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:603)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
	... 47 more


FAILED:  org.apache.solr.handler.TestReqParamsAPI.test

Error Message:
Could not get expected value  'CY val' for path 'response/params/y/c' full output: {   "responseHeader":{     "status":0,     "QTime":0},   "response":{     "znodeVersion":0,     "params":{"x":{         "a":"A val",         "b":"B val",         "":{"v":0}}}}},  from server:  https://127.0.0.1:37395/collection1

Stack Trace:
java.lang.AssertionError: Could not get expected value  'CY val' for path 'response/params/y/c' full output: {
  "responseHeader":{
    "status":0,
    "QTime":0},
  "response":{
    "znodeVersion":0,
    "params":{"x":{
        "a":"A val",
        "b":"B val",
        "":{"v":0}}}}},  from server:  https://127.0.0.1:37395/collection1
	at __randomizedtesting.SeedInfo.seed([4C17D787E792729C:C443E85D496E1F64]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
	at org.apache.solr.handler.TestReqParamsAPI.testReqParams(TestReqParamsAPI.java:159)
	at org.apache.solr.handler.TestReqParamsAPI.test(TestReqParamsAPI.java:61)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11895 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160807_105930_686.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid21736.hprof ...
   [junit4] Heap dump file created [712416286 bytes in 6.233 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160807_105930_686.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J1: EOF ----

[...truncated 46 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/init-core-data-001
   [junit4]   2> 1995750 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[4C17D787E792729C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 1995750 INFO  (SUITE-LeaderFailoverAfterPartitionTest-seed#[4C17D787E792729C]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /hd_om/
   [junit4]   2> 1995754 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1995755 INFO  (Thread-19798) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1995755 INFO  (Thread-19798) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1995855 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.ZkTestServer start zk server on port:36677
   [junit4]   2> 1995855 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1995855 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1995858 INFO  (zkCallback-1867-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@674feee7 name:ZooKeeperConnection Watcher:127.0.0.1:36677 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1995859 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1995859 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1995859 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 1995861 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1995865 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1995865 INFO  (zkCallback-1868-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1eb7aec8 name:ZooKeeperConnection Watcher:127.0.0.1:36677/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1995865 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1995865 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1995865 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 1995866 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 1995868 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 1995868 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 1995870 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1995870 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 1995872 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1995872 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 1995873 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1995874 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1995875 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1995875 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 1995876 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1995876 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 1995877 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1995877 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 1995878 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1995878 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 1995879 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1995880 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1995881 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1995881 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1995882 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1995882 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 1995883 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1995883 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 1995952 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1
   [junit4]   2> 1995954 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1995956 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f27f9f{/hd_om,null,AVAILABLE}
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@62d198b4{HTTP/1.1,[http/1.1]}{127.0.0.1:53562}
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.Server Started @1998175ms
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/tempDir-001/control/data, hostContext=/hd_om, hostPort=52303, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores}
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001'
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1995964 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1995965 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1995969 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1995970 INFO  (zkCallback-1869-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@6d70da7d name:ZooKeeperConnection Watcher:127.0.0.1:36677/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1995970 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1995970 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1995971 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1995971 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/solr.xml
   [junit4]   2> 1995975 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores
   [junit4]   2> 1995975 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.CoreContainer New CoreContainer 1477612659
   [junit4]   2> 1995975 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001]
   [junit4]   2> 1995975 WARN  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/lib
   [junit4]   2> 1995975 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 1995976 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 1995976 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1995976 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1995976 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36677/solr
   [junit4]   2> 1995976 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 1995977 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1995977 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1995978 INFO  (zkCallback-1872-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@32410b62 name:ZooKeeperConnection Watcher:127.0.0.1:36677 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1995979 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1995979 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1995980 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$96/145199802@392e3620
   [junit4]   2> 1995981 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1995981 INFO  (zkCallback-1873-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@28c3d5d3 name:ZooKeeperConnection Watcher:127.0.0.1:36677/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1995981 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1995982 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 1995983 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 1995985 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 1995986 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 1995987 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 1995987 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 1995988 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 1995989 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 1995989 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 1995990 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1995990 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 1995990 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1995992 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 1995992 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 1995993 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 1995993 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96375247254716420-127.0.0.1:52303_hd_om-n_0000000000
   [junit4]   2> 1995993 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52303_hd_om
   [junit4]   2> 1995993 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 1995994 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer Overseer (id=96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) starting
   [junit4]   2> 1995995 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 1995997 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 1995998 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 1995998 INFO  (OverseerCollectionConfigSetProcessor-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 1996002 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52303_hd_om
   [junit4]   2> 1996002 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:52303_hd_om
   [junit4]   2> 1996002 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 1996002 INFO  (zkCallback-1873-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 1996005 INFO  (zkCallback-1873-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1996015 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 1996015 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 1996015 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores
   [junit4]   2> 1996016 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 1996016 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1
   [junit4]   2> 1996016 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 1996016 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 1996016 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1996017 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 1996017 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [n:127.0.0.1:52303_hd_om    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 1996017 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 1996017 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52303/hd_om",
   [junit4]   2>   "node_name":"127.0.0.1:52303_hd_om",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1996018 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52303/hd_om",
   [junit4]   2>   "node_name":"127.0.0.1:52303_hd_om",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1996018 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 1996019 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 1996019 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 1996020 INFO  (zkCallback-1873-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1996025 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 1996025 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 1996025 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 1996026 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 1996026 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1996026 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1996026 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1'
   [junit4]   2> 1996026 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1996027 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1996028 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 1996033 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 1996037 WARN  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 1996037 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 1996044 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 1996048 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1996125 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 1996131 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 1996132 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 1996143 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1996145 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 1996146 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1996147 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 1996148 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 1996148 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 1996148 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1996148 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/]
   [junit4]   2> 1996148 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@18577059
   [junit4]   2> 1996149 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/snapshot_metadata
   [junit4]   2> 1996149 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 1996149 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data
   [junit4]   2> 1996149 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/index/
   [junit4]   2> 1996149 WARN  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1996149 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/index
   [junit4]   2> 1996150 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=46, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4530217309050658]
   [junit4]   2> 1996150 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@4a4578ae lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2bb91bf0),segFN=segments_1,generation=1}
   [junit4]   2> 1996150 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1996150 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/
   [junit4]   2> 1996150 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 1996154 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 1996155 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1996155 WARN  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 1996156 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1996157 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1996157 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1996158 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 1996160 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 1996160 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 1996160 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1996160 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=24, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=0.3271484375, floorSegmentMB=0.27734375, forceMergeDeletesPctAllowed=2.176793134409356, segmentsPerTier=42.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=RawDirectoryWrapper(RAMDirectory@4a4578ae lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2bb91bf0),segFN=segments_1,generation=1}
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@66463c78[collection1] main]
   [junit4]   2> 1996161 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 1996162 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 1996162 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 1996162 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1996162 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 1996163 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1996163 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 1996163 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1996163 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 1996163 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 1996164 INFO  (searcherExecutor-5063-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@66463c78[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1996164 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 1996164 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 1996164 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 1996164 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1542003956503281664
   [junit4]   2> 1996168 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 3.0ms to seed version buckets with highest version 1542003956503281664
   [junit4]   2> 1996168 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 1996169 INFO  (coreLoadExecutor-5062-thread-1-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 1996169 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:52303/hd_om collection:control_collection shard:shard1
   [junit4]   2> 1996170 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 1996171 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96375247254716420-core_node1-n_0000000000
   [junit4]   2> 1996171 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 1996172 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1996172 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1996172 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52303/hd_om/collection1/
   [junit4]   2> 1996172 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 1996172 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1996173 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:52303/hd_om/collection1/ has no replicas
   [junit4]   2> 1996173 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 1996175 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96375247254716420-core_node1-n_0000000000
   [junit4]   2> 1996176 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52303/hd_om/collection1/ shard1
   [junit4]   2> 1996181 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:52303/hd_om",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 1996283 INFO  (zkCallback-1873-thread-3-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1996327 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:52303/hd_om/collection1/ and leader is http://127.0.0.1:52303/hd_om/collection1/
   [junit4]   2> 1996327 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:52303/hd_om
   [junit4]   2> 1996328 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1996328 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 1996328 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 1996329 INFO  (coreZkRegister-5056-thread-1-processing-n:127.0.0.1:52303_hd_om x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:52303_hd_om c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 1996330 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52303/hd_om",
   [junit4]   2>   "node_name":"127.0.0.1:52303_hd_om",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 1996330 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:52303/hd_om",
   [junit4]   2>   "node_name":"127.0.0.1:52303_hd_om",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 1996432 INFO  (zkCallback-1873-thread-3-processing-n:127.0.0.1:52303_hd_om) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1996520 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1996520 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1996521 INFO  (zkCallback-1877-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33f25bb7 name:ZooKeeperConnection Watcher:127.0.0.1:36677/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1996521 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1996522 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1996522 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 1996522 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 1996525 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1996526 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1996526 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 1996526 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1996526 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1996527 INFO  (zkCallback-1878-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2dd1006 name:ZooKeeperConnection Watcher:127.0.0.1:36677/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 1996527 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1996528 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 1996529 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 1996529 INFO  (OverseerStateUpdate-96375247254716420-127.0.0.1:52303_hd_om-n_0000000000) [n:127.0.0.1:52303_hd_om    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 1996601 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/shard-1-001/cores/collection1
   [junit4]   2> 1996602 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/shard-1-001
   [junit4]   2> 1996602 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 1996604 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@736f22fe{/hd_om,null,AVAILABLE}
   [junit4]   2> 1996604 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3c7f5e50{HTTP/1.1,[http/1.1]}{127.0.0.1:34903}
   [junit4]   2> 1996604 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.e.j.s.Server Started @1998815ms
   [junit4]   2> 1996604 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/hd_om, hostPort=55743, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/shard-1-001/cores}
   [junit4]   2> 1996605 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 1996605 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.LeaderFailoverAfterPartitionTest_4C17D787E792729C-001/shard-1-001'
   [junit4]   2> 1996605 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1996605 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 1996606 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 1996606 INFO  (TEST-LeaderFailoverAfterPartitionTest.test-seed#[4C17D787E792729C]) [    ] o.a.s.

[...truncated too long message...]

s-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160807_105930_686.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160807_105930_686.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 237 minutes 24 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {README.txt=README.txt, heapdumps/java_pid21736.hprof=heapdumps/java_pid21736.hprof, solr/build/solr-core/test/temp/junit4-J2-20160807_105930_686.events=solr/build/solr-core/test/temp/junit4-J2-20160807_105930_686.events, solr/build/solr-core/test/temp/junit4-J1-20160807_105930_686.events=solr/build/solr-core/test/temp/junit4-J1-20160807_105930_686.events, solr/build/solr-core/test/temp/junit4-J0-20160807_105930_686.events=solr/build/solr-core/test/temp/junit4-J0-20160807_105930_686.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 138 - Still unstable

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/138/

3 tests failed.
FAILED:  org.apache.solr.cloud.SaslZkACLProviderTest.testSaslZkACLProvider

Error Message:
java.net.BindException: Address already in use

Stack Trace:
java.lang.RuntimeException: java.net.BindException: Address already in use
	at org.apache.solr.cloud.SaslZkACLProviderTest$SaslZkTestServer.run(SaslZkACLProviderTest.java:208)
	at org.apache.solr.cloud.SaslZkACLProviderTest.setUp(SaslZkACLProviderTest.java:76)
	at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:905)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.BindException: Address already in use
	at sun.nio.ch.Net.bind0(Native Method)
	at sun.nio.ch.Net.bind(Net.java:433)
	at sun.nio.ch.Net.bind(Net.java:425)
	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
	at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:252)
	at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:49)
	at org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:525)
	at org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$200(AbstractPollingIoAcceptor.java:67)
	at org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:409)
	at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	... 1 more


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.SaslZkACLProviderTest

Error Message:
6 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest:     1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at java.util.TimerThread.mainLoop(Timer.java:526)         at java.util.TimerThread.run(Timer.java:505)    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 
   1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.util.TimerThread.mainLoop(Timer.java:526)
        at java.util.TimerThread.run(Timer.java:505)
   2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([C8B0839AB9FE4F20]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.SaslZkACLProviderTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at java.util.TimerThread.mainLoop(Timer.java:526)         at java.util.TimerThread.run(Timer.java:505)    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:502)
        at java.util.TimerThread.mainLoop(Timer.java:526)
        at java.util.TimerThread.run(Timer.java:505)
   2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([C8B0839AB9FE4F20]:0)




Build Log:
[...truncated 11465 lines...]
   [junit4] Suite: org.apache.solr.cloud.SaslZkACLProviderTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SaslZkACLProviderTest_C8B0839AB9FE4F20-001/init-core-data-001
   [junit4]   2> 966786 INFO  (SUITE-SaslZkACLProviderTest-seed#[C8B0839AB9FE4F20]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 966793 INFO  (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[C8B0839AB9FE4F20]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSaslZkACLProvider
   [junit4]   2> 966793 INFO  (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[C8B0839AB9FE4F20]) [    ] o.a.s.c.SaslZkACLProviderTest ####SETUP_START testSaslZkACLProvider
   [junit4]   2> 966793 INFO  (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[C8B0839AB9FE4F20]) [    ] o.a.s.c.SaslZkACLProviderTest ZooKeeper dataDir:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SaslZkACLProviderTest_C8B0839AB9FE4F20-001/tempDir-002/zookeeper/server1/data
   [junit4]   2> 971514 WARN  (TEST-SaslZkACLProviderTest.testSaslZkACLProvider-seed#[C8B0839AB9FE4F20]) [    ] o.a.d.s.c.DefaultDirectoryService You didn't change the admin password of directory service instance 'DefaultKrbServer'.  Please update the admin password as soon as possible to prevent a possible security breach.
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SaslZkACLProviderTest -Dtests.method=testSaslZkACLProvider -Dtests.seed=C8B0839AB9FE4F20 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=no-NO -Dtests.timezone=Africa/Porto-Novo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   5.21s J2 | SaslZkACLProviderTest.testSaslZkACLProvider <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: java.net.BindException: Address already in use
   [junit4]    > 	at org.apache.solr.cloud.SaslZkACLProviderTest$SaslZkTestServer.run(SaslZkACLProviderTest.java:208)
   [junit4]    > 	at org.apache.solr.cloud.SaslZkACLProviderTest.setUp(SaslZkACLProviderTest.java:76)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]    > Caused by: java.net.BindException: Address already in use
   [junit4]    > 	at sun.nio.ch.Net.bind0(Native Method)
   [junit4]    > 	at sun.nio.ch.Net.bind(Net.java:433)
   [junit4]    > 	at sun.nio.ch.Net.bind(Net.java:425)
   [junit4]    > 	at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
   [junit4]    > 	at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
   [junit4]    > 	at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:252)
   [junit4]    > 	at org.apache.mina.transport.socket.nio.NioSocketAcceptor.open(NioSocketAcceptor.java:49)
   [junit4]    > 	at org.apache.mina.core.polling.AbstractPollingIoAcceptor.registerHandles(AbstractPollingIoAcceptor.java:525)
   [junit4]    > 	at org.apache.mina.core.polling.AbstractPollingIoAcceptor.access$200(AbstractPollingIoAcceptor.java:67)
   [junit4]    > 	at org.apache.mina.core.polling.AbstractPollingIoAcceptor$Acceptor.run(AbstractPollingIoAcceptor.java:409)
   [junit4]    > 	at org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    > 	... 1 moreThrowable #2: java.lang.NullPointerException
   [junit4]    > 	at org.apache.solr.cloud.ZkTestServer$ZKServerMain.shutdown(ZkTestServer.java:334)
   [junit4]    > 	at org.apache.solr.cloud.ZkTestServer.shutdown(ZkTestServer.java:526)
   [junit4]    > 	at org.apache.solr.cloud.SaslZkACLProviderTest$SaslZkTestServer.shutdown(SaslZkACLProviderTest.java:215)
   [junit4]    > 	at org.apache.solr.cloud.SaslZkACLProviderTest.tearDown(SaslZkACLProviderTest.java:112)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 971999 INFO  (SUITE-SaslZkACLProviderTest-seed#[C8B0839AB9FE4F20]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.SaslZkACLProviderTest_C8B0839AB9FE4F20-001
   [junit4]   2> Aug 06, 2016 11:11:27 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 6 leaked thread(s).
   [junit4]   2> Aug 06, 2016 11:11:47 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 6 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 
   [junit4]   2>    1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Object.wait(Object.java:502)
   [junit4]   2>         at java.util.TimerThread.mainLoop(Timer.java:526)
   [junit4]   2>         at java.util.TimerThread.run(Timer.java:505)
   [junit4]   2>    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]   2>         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]   2>         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Aug 06, 2016 11:11:47 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2> Aug 06, 2016 11:11:50 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at java.lang.Object.wait(Native Method)
   [junit4]   2>         at java.lang.Object.wait(Object.java:502)
   [junit4]   2>         at java.util.TimerThread.mainLoop(Timer.java:526)
   [junit4]   2>         at java.util.TimerThread.run(Timer.java:505)
   [junit4]   2>    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]   2>         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]   2>         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2>    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=6, maxDocsPerChunk=5, blockSize=700), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=6, blockSize=700)), sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=no-NO, timezone=Africa/Porto-Novo
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=7,free=147174304,total=531628032
   [junit4]   2> NOTE: All tests run in this JVM: [LukeRequestHandlerTest, NoCacheHeaderTest, TestFilteredDocIdSet, RestartWhileUpdatingTest, TestAuthorizationFramework, TestBackupRepositoryFactory, TestSolrFieldCacheMBean, CircularListTest, CollectionStateFormat2Test, HardAutoCommitTest, StressHdfsTest, XmlUpdateRequestHandlerTest, TestCustomDocTransformer, SolrTestCaseJ4Test, TestLegacyFieldCache, BitVectorTest, TestPHPSerializedResponseWriter, StatelessScriptUpdateProcessorFactoryTest, TestRawResponseWriter, TestSubQueryTransformerCrossCore, TestSolrIndexConfig, TestTestInjection, SolrCoreCheckLockOnStartupTest, TestHdfsCloudBackupRestore, CurrencyFieldXmlFileTest, CleanupOldIndexTest, TestDFISimilarityFactory, AutoCommitTest, TestInitQParser, TestSortByMinMaxFunction, TestRecovery, TolerantUpdateProcessorTest, TestRecoveryHdfs, TestMiniSolrCloudClusterBase, TestDistributedMissingSort, DeleteShardTest, TestExtendedDismaxParser, TestStressRecovery, TestOnReconnectListenerSupport, TestNoOpRegenerator, TestManagedResourceStorage, UnloadDistributedZkTest, SuggestComponentTest, TestSort, TestManagedSynonymFilterFactory, FacetPivotSmallTest, TestCollationFieldDocValues, TestJmxIntegration, SliceStateTest, TestConfigSetProperties, SuggesterTest, TestCoreDiscovery, SharedFSAutoReplicaFailoverUtilsTest, AnalyticsMergeStrategyTest, DistributedQueryComponentCustomSortTest, TestJettySolrRunner, TestZkChroot, LeaderFailoverAfterPartitionTest, TestSimpleQParserPlugin, TestSchemaNameResource, DocValuesMultiTest, TestSolrConfigHandler, OverseerRolesTest, MergeStrategyTest, CheckHdfsIndexTest, TestReqParamsAPI, SaslZkACLProviderTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SaslZkACLProviderTest -Dtests.seed=C8B0839AB9FE4F20 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=no-NO -Dtests.timezone=Africa/Porto-Novo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J2 | SaslZkACLProviderTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 6 threads leaked from SUITE scope at org.apache.solr.cloud.SaslZkACLProviderTest: 
   [junit4]    >    1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.lang.Object.wait(Object.java:502)
   [junit4]    >         at java.util.TimerThread.mainLoop(Timer.java:526)
   [junit4]    >         at java.util.TimerThread.run(Timer.java:505)
   [junit4]    >    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([C8B0839AB9FE4F20]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=15650, name=apacheds, state=WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at java.lang.Object.wait(Object.java:502)
   [junit4]    >         at java.util.TimerThread.mainLoop(Timer.java:526)
   [junit4]    >         at java.util.TimerThread.run(Timer.java:505)
   [junit4]    >    2) Thread[id=15654, name=groupCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    3) Thread[id=15655, name=pool-54-thread-1, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    4) Thread[id=15652, name=ou=system.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    5) Thread[id=15653, name=kdcReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    6) Thread[id=15651, name=changePwdReplayCache.data, state=TIMED_WAITING, group=TGRP-SaslZkACLProviderTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([C8B0839AB9FE4F20]:0)
   [junit4] Completed [125/626 (1!)] on J2 in 28.53s, 1 test, 3 errors <<< FAILURES!

[...truncated 61614 lines...]



[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 137 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/137/

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([CC4C799E8C562B1C]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11732 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/init-core-data-001
   [junit4]   2> 809140 INFO  (SUITE-TestReplicationHandler-seed#[CC4C799E8C562B1C]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4] IGNOR/A 0.00s J0 | TestReplicationHandler.doTestIndexFetchOnMasterRestart
   [junit4]    > Assumption #1: 'awaitsfix' test group is disabled (@AwaitsFix(bugUrl=https://issues.apache.org/jira/browse/SOLR-9036))
   [junit4]   2> 809142 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestReplicateAfterStartup
   [junit4]   2> 809142 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1
   [junit4]   2> 809144 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 809146 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4d34093{/solr,null,AVAILABLE}
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@3e4334b6{HTTP/1.1,[http/1.1]}{127.0.0.1:56861}
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.Server Started @811759ms
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=56861}
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001'
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 809147 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 809148 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/solr.xml
   [junit4]   2> 809151 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/.
   [junit4]   2> 809151 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer New CoreContainer 1942010471
   [junit4]   2> 809151 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001]
   [junit4]   2> 809151 WARN  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/lib
   [junit4]   2> 809152 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 809153 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 809153 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 809153 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 809153 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 809153 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 809153 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/.
   [junit4]   2> 809154 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 809154 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1
   [junit4]   2> 809154 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 809158 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1'
   [junit4]   2> 809158 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 809159 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 809159 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 809159 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 809163 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 809165 WARN  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 809165 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 809170 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 809171 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 809173 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 809174 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1'
   [junit4]   2> 809174 INFO  (coreLoadExecutor-1559-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1
   [junit4]   2> 809174 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/snapshot_metadata
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 809175 WARN  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 809175 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 809176 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@253a50c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6f5a06ec; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 809176 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 809176 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/
   [junit4]   2> 809177 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 809178 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809179 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809179 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809180 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809180 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 809180 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,lazy,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 809180 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=24.6630859375, floorSegmentMB=1.19140625, forceMergeDeletesPctAllowed=23.02471689484807, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4020035669670132
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@253a50c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6f5a06ec; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@42e0fd9f[collection1] main]
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/conf
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/conf}
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/conf
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 809181 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 809182 INFO  (searcherExecutor-1560-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@42e0fd9f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 809182 INFO  (coreLoadExecutor-1559-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 809660 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1
   [junit4]   2> 809662 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 809663 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5898ea8b{/solr,null,AVAILABLE}
   [junit4]   2> 809663 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@deef306{HTTP/1.1,[http/1.1]}{127.0.0.1:38139}
   [junit4]   2> 809663 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.Server Started @812275ms
   [junit4]   2> 809663 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=38139}
   [junit4]   2> 809664 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 809664 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002'
   [junit4]   2> 809664 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 809664 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 809664 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/solr.xml
   [junit4]   2> 809667 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/.
   [junit4]   2> 809667 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer New CoreContainer 1817000418
   [junit4]   2> 809667 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002]
   [junit4]   2> 809668 WARN  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/lib
   [junit4]   2> 809668 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 809669 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 809669 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 809669 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 809669 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 809669 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 809669 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/.
   [junit4]   2> 809670 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 809670 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1
   [junit4]   2> 809670 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 809670 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1'
   [junit4]   2> 809671 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 809671 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 809671 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 809671 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 809674 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 809682 WARN  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 809683 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 809687 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 809689 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 809690 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 809691 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1'
   [junit4]   2> 809691 INFO  (coreLoadExecutor-1569-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1
   [junit4]   2> 809691 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/snapshot_metadata
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 809692 WARN  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 809692 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 809693 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@77545013 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7bf82d3d; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 809693 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 809693 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/
   [junit4]   2> 809694 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 809695 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809695 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809696 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809697 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 809697 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 809697 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,lazy,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=24.6630859375, floorSegmentMB=1.19140625, forceMergeDeletesPctAllowed=23.02471689484807, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4020035669670132
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@77545013 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@7bf82d3d; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7fbd81b2[collection1] main]
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1/conf
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1/conf}
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/collection1/conf
   [junit4]   2> 809698 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 809700 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 809700 INFO  (searcherExecutor-1570-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@7fbd81b2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 809700 INFO  (coreLoadExecutor-1569-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 809902 INFO  (qtp1677376140-50770) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 810176 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@deef306{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 810176 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1817000418
   [junit4]   2> 810177 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@284f9677
   [junit4]   2> 810177 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 810177 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 810177 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 810177 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 3 directories currently being tracked
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/snapshot_metadata [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/snapshot_metadata;done=false>>]
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/snapshot_metadata
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data;done=false>>]
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/index;done=false>>]
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 810178 INFO  (coreCloseExecutor-1575-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 810179 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5898ea8b{/solr,null,UNAVAILABLE}
   [junit4]   2> 810181 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 810181 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
   [junit4]   2> 810184 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 810184 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@253a50c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6f5a06ec; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@253a50c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6f5a06ec; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_2,generation=2}
   [junit4]   2> 810184 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 2
   [junit4]   2> 810184 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@445ca1a5[collection1] main]
   [junit4]   2> 810184 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 810184 INFO  (searcherExecutor-1560-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@445ca1a5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 810185 INFO  (qtp1677376140-50773) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 1
   [junit4]   2> 810191 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@3e4334b6{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 810191 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1942010471
   [junit4]   2> 810196 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4b506091
   [junit4]   2> 810196 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 3 directories currently being tracked
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data;done=false>>]
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index;done=false>>]
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/snapshot_metadata [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/snapshot_metadata;done=false>>]
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 810197 INFO  (coreCloseExecutor-1577-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/snapshot_metadata
   [junit4]   2> 810202 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4d34093{/solr,null,UNAVAILABLE}
   [junit4]   2> 810203 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 810204 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@20ad5f7f{/solr,null,AVAILABLE}
   [junit4]   2> 810204 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.ServerConnector Started ServerConnector@6c4d4310{HTTP/1.1,[http/1.1]}{127.0.0.1:33173}
   [junit4]   2> 810204 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.e.j.s.Server Started @812817ms
   [junit4]   2> 810205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=33173}
   [junit4]   2> 810205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 810205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001'
   [junit4]   2> 810205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 810205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 810205 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/solr.xml
   [junit4]   2> 810210 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/.
   [junit4]   2> 810210 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer New CoreContainer 650691908
   [junit4]   2> 810210 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001]
   [junit4]   2> 810210 WARN  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/lib
   [junit4]   2> 810210 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 810221 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 810225 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 810225 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 810227 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 810227 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 810227 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/.
   [junit4]   2> 810228 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 810228 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1
   [junit4]   2> 810228 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 810230 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1'
   [junit4]   2> 810237 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 810237 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 810231 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 810244 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartup-seed#[CC4C799E8C562B1C]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 810247 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 810249 WARN  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 810249 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 810254 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 810256 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 810260 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1'
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/snapshot_metadata
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data
   [junit4]   2> 810261 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 810261 WARN  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 810262 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 810262 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 810262 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@4f1b47fc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6dd00a54; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 810262 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 810262 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/./collection1/data/
   [junit4]   2> 810263 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 810264 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 810264 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 810265 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 810265 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,lazy,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=47, maxMergedSegmentMB=24.6630859375, floorSegmentMB=1.19140625, forceMergeDeletesPctAllowed=23.02471689484807, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4020035669670132
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@4f1b47fc lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6dd00a54; maxCacheMB=0.87109375 maxMergeSizeMB=0.3525390625)),segFN=segments_1,generation=1}
   [junit4]   2> 810266 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@40414c6e[collection1] main]
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/conf
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/conf}
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_CC4C799E8C562B1C-001/solr-instance-001/collection1/conf
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 810267 INFO  (coreLoadExecutor-1584-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 810267 INFO  (searcherExecutor-1585-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@40414c6e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 810750 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 3
   [junit4]   2> 810751 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
   [junit4]   2> 810753 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
   [junit4]   2> 810754 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
   [junit4]   2> 810756 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
   [junit4]   2> 810757 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
   [junit4]   2> 810759 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
   [junit4]   2> 810760 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
   [junit4]   2> 810762 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
   [junit4]   2> 810763 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
   [junit4]   2> 810764 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
   [junit4]   2> 810766 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
   [junit4]   2> 810767 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
   [junit4]   2> 810769 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
   [junit4]   2> 810770 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
   [junit4]   2> 810771 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
   [junit4]   2> 810773 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
   [junit4]   2> 810774 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
   [junit4]   2> 810776 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
   [junit4]   2> 810777 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
   [junit4]   2> 810779 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
   [junit4]   2> 810780 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
   [junit4]   2> 810781 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
   [junit4]   2> 810783 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
   [junit4]   2> 810784 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
   [junit4]   2> 810786 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
   [junit4]   2> 810787 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
   [junit4]   2> 810789 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
   [junit4]   2> 810790 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
   [junit4]   2> 810792 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
   [junit4]   2> 810793 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
   [junit4]   2> 810795 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
   [junit4]   2> 810798 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
   [junit4]   2> 810799 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
   [junit4]   2> 810801 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
   [junit4]   2> 810802 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
   [junit4]   2> 810803 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
   [junit4]   2> 810805 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
   [junit4]   2> 810806 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
   [junit4]   2> 810808 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
   [junit4]   2> 810809 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[40]} 0 0
   [junit4]   2> 810810 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[41]} 0 0
   [junit4]   2> 810812 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[42]} 0 0
   [junit4]   2> 810813 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[43]} 0 0
   [junit4]   2> 810815 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[44]} 0 0
   [junit4]   2> 810816 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[45]} 0 0
   [junit4]   2> 810818 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[46]} 0 0
   [junit4]   2> 810819 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[47]} 0 0
   [junit4]   2> 810821 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[48]} 0 0
   [junit4]   2> 810822 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[49]} 0 0
   [junit4]   2> 810824 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[50]} 0 0
   [junit4]   2> 810825 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[51]} 0 0
   [junit4]   2> 810826 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[52]} 0 0
   [junit4]   2> 810828 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[53]} 0 0
   [junit4]   2> 810829 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[54]} 0 0
   [junit4]   2> 810831 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[55]} 0 0
   [junit4]   2> 810832 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[56]} 0 0
   [junit4]   2> 810834 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[57]} 0 0
   [junit4]   2> 810835 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[58]} 0 0
   [junit4]   2> 810837 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[59]} 0 0
   [junit4]   2> 810838 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[60]} 0 0
   [junit4]   2> 810839 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[61]} 0 0
   [junit4]   2> 810841 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[62]} 0 0
   [junit4]   2> 810842 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[63]} 0 0
   [junit4]   2> 810844 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[64]} 0 0
   [junit4]   2> 810845 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[65]} 0 0
   [junit4]   2> 810847 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[66]} 0 0
   [junit4]   2> 810848 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[67]} 0 0
   [junit4]   2> 810850 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[68]} 0 0
   [junit4]   2> 810851 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[69]} 0 0
   [junit4]   2> 810853 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[70]} 0 0
   [junit4]   2> 810855 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[71]} 0 0
   [junit4]   2> 810856 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[72]} 0 0
   [junit4]   2> 810858 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[73]} 0 0
   [junit4]   2> 810859 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[74]} 0 0
   [junit4]   2> 810860 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[75]} 0 0
   [junit4]   2> 810862 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[76]} 0 0
   [junit4]   2> 810863 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[77]} 0 0
   [junit4]   2> 810865 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[78]} 0 0
   [junit4]   2> 810866 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[79]} 0 0
   [junit4]   2> 810868 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[80]} 0 0
   [junit4]   2> 810869 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[81]} 0 0
   [junit4]   2> 810871 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[82]} 0 0
   [junit4]   2> 810872 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[83]} 0 0
   [junit4]   2> 810874 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[84]} 0 0
   [junit4]   2> 810875 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[85]} 0 0
   [junit4]   2> 810877 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[86]} 0 0
   [junit4]   2> 810878 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[87]} 0 0
   [junit4]   2> 810880 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[88]} 0 0
   [junit4]   2> 810881 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[89]} 0 0
   [junit4]   2> 810883 INFO  (qtp990030861-50808) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[90]} 0 0
   [junit4]   2> 810884 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[91]} 0 0
   [junit4]   2> 810886 INFO  (qtp990030861-50811) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[92]} 0 0
   [junit4]   2> 810887 INFO  (qtp990030861-50810) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[93]} 0 0
   [junit4]   2> 810889 INFO  (qtp990030861-50805) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[94]} 0 0
   [junit4]   2> 810890 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[95]} 0 0
   [junit4]   2> 810892 INFO  (qtp990030861-50812) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[96]} 0 0
   [junit4]   2> 810893 INFO  (qtp990030861-50809) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[97]} 0 0
   [junit4]   2> 810894 INFO  (qtp990030861-50811)

[...truncated too long message...]

15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160805_152100_908.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160805_152100_908.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 290 minutes 18 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {heapdumps/java_pid24995.hprof=heapdumps/java_pid24995.hprof, README.txt=README.txt, heapdumps/java_pid24984.hprof=heapdumps/java_pid24984.hprof, solr/build/solr-core/test/temp/junit4-J1-20160805_152100_908.events=solr/build/solr-core/test/temp/junit4-J1-20160805_152100_908.events, solr/build/solr-core/test/temp/junit4-J2-20160805_152100_908.events=solr/build/solr-core/test/temp/junit4-J2-20160805_152100_908.events, solr/build/solr-core/test/temp/junit4-J0-20160805_152100_908.events=solr/build/solr-core/test/temp/junit4-J0-20160805_152100_908.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 136 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/136/

5 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest:     1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([447D65A31375D152]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.CollectionsAPIDistributedZkTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
        at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([447D65A31375D152]:0)


FAILED:  org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test

Error Message:
Timeout waiting for all live and active

Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
	at __randomizedtesting.SeedInfo.seed([447D65A31375D152:CC295A79BD89BCAA]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:309)
	at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:127)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.cloud.hdfs.HdfsCollectionsAPIDistributedZkTest.test

Error Message:
Captured an uncaught exception in thread: Thread[id=12467, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=12467, name=collection1, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:57130: collection already exists: awholynewstresscollection_collection1_3
	at __randomizedtesting.SeedInfo.seed([447D65A31375D152]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:592)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:403)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:356)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1291)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:1061)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:997)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1599)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1620)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest$1CollectionThread.run(CollectionsAPIDistributedZkTest.java:987)


FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:40237/fl_x/t

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:40237/fl_x/t
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:603)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:457)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
	... 47 more




Build Log:
[...truncated 11365 lines...]
   [junit4] Suite: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/init-core-data-001
   [junit4]   2> 254073 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4]   2> 254073 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /_fps/vg
   [junit4]   2> 254528 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 255009 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
   [junit4]   2> 255140 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
   [junit4]   2> 255161 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 255209 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 255242 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_50023_hdfs____ek1uow/webapp
   [junit4]   2> 255891 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:50023
   [junit4]   2> 256456 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 256460 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 256471 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_51821_datanode____.84rgou/webapp
   [junit4]   2> 256819 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51821
   [junit4]   2> 257081 WARN  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 257082 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log jetty-6.1.26
   [junit4]   2> 257091 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.2-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_32782_datanode____rtwux/webapp
   [junit4]   2> 257494 INFO  (SUITE-SharedFSAutoReplicaFailoverTest-seed#[447D65A31375D152]-worker) [    ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:32782
   [junit4]   2> 258136 INFO  (IPC Server handler 8 on 37901) [    ] BlockStateChange BLOCK* processReport: from storage DS-a8f5c190-63b3-4629-807f-fc05a04e0d4b node DatanodeRegistration(127.0.0.1:58374, datanodeUuid=80f943f2-4820-44f7-bf0e-2f90fe38385e, infoPort=45592, infoSecurePort=0, ipcPort=42985, storageInfo=lv=-56;cid=testClusterID;nsid=745202128;c=0), blocks: 0, hasStaleStorage: true, processing time: 10 msecs
   [junit4]   2> 258136 INFO  (IPC Server handler 8 on 37901) [    ] BlockStateChange BLOCK* processReport: from storage DS-6453c9b3-e4b7-46f6-bc25-0d3bd7b2f588 node DatanodeRegistration(127.0.0.1:58374, datanodeUuid=80f943f2-4820-44f7-bf0e-2f90fe38385e, infoPort=45592, infoSecurePort=0, ipcPort=42985, storageInfo=lv=-56;cid=testClusterID;nsid=745202128;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 258143 INFO  (IPC Server handler 9 on 37901) [    ] BlockStateChange BLOCK* processReport: from storage DS-6bbbc032-999f-4e07-9db6-4db3190e8013 node DatanodeRegistration(127.0.0.1:46885, datanodeUuid=be6f71d1-6f3a-4256-9a8b-1400ad64c176, infoPort=54972, infoSecurePort=0, ipcPort=60610, storageInfo=lv=-56;cid=testClusterID;nsid=745202128;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
   [junit4]   2> 258143 INFO  (IPC Server handler 9 on 37901) [    ] BlockStateChange BLOCK* processReport: from storage DS-f0b2f890-22e8-4534-91c2-2b3547b5cddd node DatanodeRegistration(127.0.0.1:46885, datanodeUuid=be6f71d1-6f3a-4256-9a8b-1400ad64c176, infoPort=54972, infoSecurePort=0, ipcPort=60610, storageInfo=lv=-56;cid=testClusterID;nsid=745202128;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
   [junit4]   2> 258443 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 258443 INFO  (Thread-834) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 258443 INFO  (Thread-834) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 258543 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.ZkTestServer start zk server on port:44085
   [junit4]   2> 258544 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 258544 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 258547 INFO  (zkCallback-521-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1f670e2 name:ZooKeeperConnection Watcher:127.0.0.1:44085 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 258547 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 258547 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 258547 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 258550 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 258550 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 258551 INFO  (zkCallback-522-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2cd37c90 name:ZooKeeperConnection Watcher:127.0.0.1:44085/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 258551 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 258551 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 258551 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 258559 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 258562 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 258563 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 258563 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 258564 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 258565 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 258565 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 258566 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 258566 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 258567 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 258567 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 258568 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 258568 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 258569 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 258569 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 258570 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 258570 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 258570 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 258570 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 258571 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 258571 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 258572 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 258572 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 258573 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 258573 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 258634 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores/collection1
   [junit4]   2> 258635 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 258636 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@56d4dd3b{/_fps/vg,null,AVAILABLE}
   [junit4]   2> 258636 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.e.j.s.ServerConnector Started ServerConnector@60ada1ca{HTTP/1.1,[http/1.1]}{127.0.0.1:53020}
   [junit4]   2> 258636 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.e.j.s.Server Started @260903ms
   [junit4]   2> 258636 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/_fps/vg, hostPort=53020, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores}
   [junit4]   2> 258637 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 258637 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001'
   [junit4]   2> 258637 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 258637 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 258637 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 258637 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 258639 INFO  (zkCallback-523-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@470f3bf7 name:ZooKeeperConnection Watcher:127.0.0.1:44085/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 258639 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 258639 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 258640 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 258640 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/solr.xml
   [junit4]   2> 258644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores
   [junit4]   2> 258644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.CoreContainer New CoreContainer 1942926815
   [junit4]   2> 258644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001]
   [junit4]   2> 258644 WARN  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/lib
   [junit4]   2> 258644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 258655 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 258656 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 258656 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 258656 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44085/solr
   [junit4]   2> 258656 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 258656 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 258657 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 258657 INFO  (zkCallback-526-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@53cd312e name:ZooKeeperConnection Watcher:127.0.0.1:44085 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 258658 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 258658 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 258659 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$23/1149697902@4ab3c19d
   [junit4]   2> 258660 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 258660 INFO  (zkCallback-527-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@10e10a8c name:ZooKeeperConnection Watcher:127.0.0.1:44085/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 258660 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 258661 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 258661 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 258662 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 258663 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 258664 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 258665 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 258665 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 258666 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 258666 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 258667 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 258667 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 258667 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 258668 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 258669 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 258671 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 258671 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000
   [junit4]   2> 258672 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:53020__fps%2Fvg
   [junit4]   2> 258672 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 258672 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer Overseer (id=96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) starting
   [junit4]   2> 258673 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 258676 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 258677 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 258677 INFO  (OverseerCollectionConfigSetProcessor-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 258680 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53020__fps%2Fvg
   [junit4]   2> 258680 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:53020__fps%2Fvg
   [junit4]   2> 258681 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 258682 INFO  (zkCallback-527-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 258682 INFO  (zkCallback-527-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 258691 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 258691 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 258691 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores
   [junit4]   2> 258692 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 258692 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores/collection1
   [junit4]   2> 258692 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 258693 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 258693 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 258693 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 258693 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 258693 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 258693 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53020/_fps/vg",
   [junit4]   2>   "node_name":"127.0.0.1:53020__fps%2Fvg",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 258694 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53020/_fps/vg",
   [junit4]   2>   "node_name":"127.0.0.1:53020__fps%2Fvg",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 258694 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 258694 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 258694 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 258695 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 258697 INFO  (zkCallback-527-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 259695 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 259695 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 259695 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 259695 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 259696 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 259696 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores/collection1'
   [junit4]   2> 259696 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 259696 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 259697 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 259702 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 259705 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 259712 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 259715 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 259797 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 259798 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 259798 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 259800 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 259817 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1
   [junit4]   2> 259818 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 259818 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.HdfsDirectoryFactory
   [junit4]   2> 259825 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:37901/solr_hdfs_home
   [junit4]   2> 259825 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 259825 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.SharedFSAutoReplicaFailoverTest_447D65A31375D152-001/control-001/cores/collection1], dataDir=[hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/]
   [junit4]   2> 259825 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@3ee102d8
   [junit4]   2> 259825 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 259838 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 259838 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 259839 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 261722 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 261722 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/snapshot_metadata
   [junit4]   2> 261723 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 261728 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 261742 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data
   [junit4]   2> 261744 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/index/
   [junit4]   2> 261758 WARN  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory 'hdfs:/localhost:37901/solr_hdfs_home/control_collection/core_node1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 261758 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 261764 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 261764 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [8388608] will allocate [1] slabs and use ~[8388608] bytes
   [junit4]   2> 261764 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 261774 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 261774 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/index
   [junit4]   2> 261775 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=26, maxMergeAtOnceExplicit=43, maxMergedSegmentMB=87.306640625, floorSegmentMB=1.6142578125, forceMergeDeletesPctAllowed=1.726215935344575, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 261974 INFO  (IPC Server handler 1 on 37901) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:58374 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-6453c9b3-e4b7-46f6-bc25-0d3bd7b2f588:NORMAL:127.0.0.1:58374|RBW], ReplicaUC[[DISK]DS-6bbbc032-999f-4e07-9db6-4db3190e8013:NORMAL:127.0.0.1:46885|RBW]]} size 71
   [junit4]   2> 261975 INFO  (IPC Server handler 6 on 37901) [    ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:46885 is added to blk_1073741825_1001 size 71
   [junit4]   2> 262384 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d881e82); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 262384 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 262386 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/
   [junit4]   2> 262388 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 262388 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 262388 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 262388 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 262388 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 262389 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 262389 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 262389 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 262389 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 262389 WARN  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 262390 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 262391 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 262391 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 262392 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 262401 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 262401 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 262410 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 262410 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 262410 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 262421 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 262421 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 262423 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@35f5b7de
   [junit4]   2> 262479 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(BlockDirectory(HdfsDirectory@hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@4d881e82); maxCacheMB=192.0 maxMergeSizeMB=16.0),segFN=segments_1,generation=1}
   [junit4]   2> 262479 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 262479 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3466b80[collection1] main]
   [junit4]   2> 262479 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 262480 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 262482 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 262483 INFO  (searcherExecutor-986-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3466b80[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 262483 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 262483 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 262484 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 262484 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1541730355273269248
   [junit4]   2> 262485 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1541730355273269248
   [junit4]   2> 262485 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 262486 INFO  (coreLoadExecutor-985-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 262486 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:53020/_fps/vg collection:control_collection shard:shard1
   [junit4]   2> 262487 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 262489 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96358146939879428-core_node1-n_0000000000
   [junit4]   2> 262489 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 262490 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 262490 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 262490 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 262490 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:53020/_fps/vg/collection1/
   [junit4]   2> 262491 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 262491 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:53020/_fps/vg/collection1/ has no replicas
   [junit4]   2> 262491 INFO  (zkCallback-527-thread-2-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 262492 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 262493 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96358146939879428-core_node1-n_0000000000
   [junit4]   2> 262494 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:53020/_fps/vg/collection1/ shard1
   [junit4]   2> 262495 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:53020/_fps/vg",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 2
   [junit4]   2> 262596 INFO  (zkCallback-527-thread-2-processing-n:127.0.0.1:53020__fps%2Fvg) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 262643 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 262643 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 262644 INFO  (zkCallback-531-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@263f1b79 name:ZooKeeperConnection Watcher:127.0.0.1:44085/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 262644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 262644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 262644 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 262645 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 262645 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 262646 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 262646 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 262646 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 262646 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:53020/_fps/vg/collection1/ and leader is http://127.0.0.1:53020/_fps/vg/collection1/
   [junit4]   2> 262646 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:53020/_fps/vg
   [junit4]   2> 262646 INFO  (TEST-SharedFSAutoReplicaFailoverTest.test-seed#[447D65A31375D152]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 262646 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 262646 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 262646 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 262647 INFO  (coreZkRegister-979-thread-1-processing-n:127.0.0.1:53020__fps%2Fvg x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:53020__fps%2Fvg c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 262647 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:53020/_fps/vg",
   [junit4]   2>   "node_name":"127.0.0.1:53020__fps%2Fvg",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state",
   [junit4]   2>   "ulogDir":"hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/tlog"} current state version: 3
   [junit4]   2> 262648 INFO  (OverseerStateUpdate-96358146939879428-127.0.0.1:53020__fps%2Fvg-n_0000000000) [n:127.0.0.1:53020__fps%2Fvg    ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "dataDir":"hdfs://localhost:37901/solr_hdfs_home/control_collection/core_node1/data/",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:5302

[...truncated too long message...]

tedZkTest-seed#[447D65A31375D152]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_447D65A31375D152-001
   [junit4]   2> Aug 04, 2016 12:11:38 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
   [junit4]   2> Aug 04, 2016 12:11:58 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   [junit4]   2>    1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Aug 04, 2016 12:11:58 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> INFO: Starting to interrupt leaked threads:
   [junit4]   2>    1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2> Aug 04, 2016 12:12:01 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
   [junit4]   2> SEVERE: There are still zombie threads that couldn't be terminated:
   [junit4]   2>    1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]   2>         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene62), sim=RandomSimilarity(queryNorm=false,coord=crazy): {}, locale=vi-VN, timezone=Mexico/BajaNorte
   [junit4]   2> NOTE: Linux 3.13.0-85-generic amd64/Oracle Corporation 1.8.0_74 (64-bit)/cpus=4,threads=2,free=149933408,total=495976448
   [junit4]   2> NOTE: All tests run in this JVM: [ZkStateReaderTest, ShardRoutingTest, ChaosMonkeySafeLeaderTest, IndexBasedSpellCheckerTest, WordBreakSolrSpellCheckerTest, ChaosMonkeyNothingIsSafeTest, DistanceUnitsTest, TermVectorComponentTest, TestReplicationHandlerBackup, MBeansHandlerTest, TestUpdate, ZkControllerTest, SharedFSAutoReplicaFailoverTest, DistributedDebugComponentTest, SuggesterFSTTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, AsyncMigrateRouteKeyTest, TestNamedUpdateProcessors, LeaderInitiatedRecoveryOnShardRestartTest, OverseerModifyCollectionTest, TestSolrXml, TestCursorMarkWithoutUniqueKey, TestMacros, DistribCursorPagingTest, TestCodecSupport, OutOfBoxZkACLAndCredentialsProvidersTest, TestFieldSortValues, ClassificationUpdateProcessorFactoryTest, TestStressUserVersions, InfoHandlerTest, TestBM25SimilarityFactory, TestFiltering, TestNumericTerms64, TimeZoneUtilsTest, TestManagedSchemaAPI, TestHashQParserPlugin, DocValuesTest, TestSearcherReuse, SolrCmdDistributorTest, TestCryptoKeys, TestQuerySenderNoQuery, SolrIndexSplitterTest, SpatialFilterTest, TestMiniSolrCloudClusterSSL, TestHashPartitioner, DOMUtilTest, ReplicationFactorTest, TriLevelCompositeIdRoutingTest, CustomCollectionTest, TestSchemaSimilarityResource, TermsComponentTest, TestJsonRequest, TestPartialUpdateDeduplication, HdfsDirectoryTest, DistributedMLTComponentTest, BlockCacheTest, HdfsRecoveryZkTest, UniqFieldsUpdateProcessorFactoryTest, DirectSolrSpellCheckerTest, TestOrdValues, TestBulkSchemaAPI, TestSolrCoreProperties, FastVectorHighlighterTest, HdfsCollectionsAPIDistributedZkTest, TestXmlQParserPlugin, TestGraphMLResponseWriter, NotRequiredUniqueKeyTest, TestShardHandlerFactory, TestRandomRequestDistribution, TestConfigSetImmutable, TestExclusionRuleCollectionAccess, TestBulkSchemaConcurrent, TestCoreContainer, TestLFUCache, TestRequestForwarding, TestTrie, TestFastLRUCache, TestCollapseQParserPlugin, TestSolrQueryResponse, MoreLikeThisHandlerTest, SuggestComponentContextFilterQueryTest, TestMiniSolrCloudCluster, CoreAdminCreateDiscoverTest, CSVRequestHandlerTest, TestQueryUtils, TestExactStatsCache, TestSubQueryTransformer, FullHLLTest, HdfsWriteToMultipleCollectionsTest, TestDistributedGrouping, OpenCloseCoreStressTest, HttpPartitionTest, HighlighterTest, MultiTermTest, TestLeaderInitiatedRecoveryThread, TestAuthenticationFramework, ConvertedLegacyTest, SpellingQueryConverterTest, TestFieldCollectionResource, CollectionsAPIDistributedZkTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CollectionsAPIDistributedZkTest -Dtests.seed=447D65A31375D152 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=vi-VN -Dtests.timezone=Mexico/BajaNorte -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | CollectionsAPIDistributedZkTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.CollectionsAPIDistributedZkTest: 
   [junit4]    >    1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([447D65A31375D152]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   [junit4]    >    1) Thread[id=25792, name=searcherExecutor-8679-thread-1, state=WAITING, group=TGRP-CollectionsAPIDistributedZkTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
   [junit4]    >         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([447D65A31375D152]:0)
   [junit4] Completed [490/626 (3!)] on J1 in 619.04s, 1 test, 3 errors <<< FAILURES!

[...truncated 404 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160804_105937_505.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid1205.hprof ...
   [junit4] Heap dump file created [685134994 bytes in 4.170 secs]
   [junit4] <<< JVM J1: EOF ----

[...truncated 11057 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:715: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid1205.hprof

Total time: 237 minutes 9 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 135 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/135/

1 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:46685

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:46685
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:603)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:261)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:250)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:457)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:497)
	... 47 more




Build Log:
[...truncated 11617 lines...]
   [junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160803_161425_766.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps/java_pid1102.hprof ...
   [junit4] Heap dump file created [664574313 bytes in 4.467 secs]
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160803_161425_766.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] WARN: Unhandled exception in event serialization. -> java.lang.OutOfMemoryError: GC overhead limit exceeded
   [junit4] <<< JVM J1: EOF ----

[...truncated 288 lines...]
   [junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/init-core-data-001
   [junit4]   2> 2723169 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[146D5B52C5AF51A]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 2723169 INFO  (SUITE-CollectionsAPIDistributedZkTest-seed#[146D5B52C5AF51A]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 2723171 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2723171 INFO  (Thread-38431) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2723171 INFO  (Thread-38431) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2723271 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ZkTestServer start zk server on port:50426
   [junit4]   2> 2723271 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723271 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723273 INFO  (zkCallback-2772-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@69a43639 name:ZooKeeperConnection Watcher:127.0.0.1:50426 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723273 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723274 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723274 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /solr
   [junit4]   2> 2723275 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723276 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723277 INFO  (zkCallback-2773-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@1eb9e519 name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723277 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723278 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1
   [junit4]   2> 2723279 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/collection1/shards
   [junit4]   2> 2723280 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection
   [junit4]   2> 2723280 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/shards
   [junit4]   2> 2723281 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 2723281 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 2723282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 2723282 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/schema.xml
   [junit4]   2> 2723283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2723283 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 2723284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 2723284 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 2723285 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 2723285 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/protwords.txt
   [junit4]   2> 2723286 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 2723286 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/currency.xml
   [junit4]   2> 2723287 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 2723287 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 2723287 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2723287 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 2723288 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2723288 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 2723289 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 2723289 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 2723290 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractZkTestCase put /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 2723290 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 2723348 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1
   [junit4]   2> 2723349 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2723350 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4cc6748e{/,null,AVAILABLE}
   [junit4]   2> 2723353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@62a9519c{HTTP/1.1,[http/1.1]}{127.0.0.1:33120}
   [junit4]   2> 2723353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.Server Started @2725578ms
   [junit4]   2> 2723353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/tempDir-001/control/data, hostContext=/, hostPort=33120, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores}
   [junit4]   2> 2723353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2723353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001'
   [junit4]   2> 2723353 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2723354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2723354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723354 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723355 INFO  (zkCallback-2774-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2bebb18b name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723355 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723356 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2723356 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/solr.xml
   [junit4]   2> 2723362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores
   [junit4]   2> 2723362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1803722666
   [junit4]   2> 2723362 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001]
   [junit4]   2> 2723362 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/lib
   [junit4]   2> 2723363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2723363 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2723364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2723364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2723364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50426/solr
   [junit4]   2> 2723364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2723364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723364 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723365 INFO  (zkCallback-2777-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@a735bd6 name:ZooKeeperConnection Watcher:127.0.0.1:50426 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723365 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$12/976932739@70189276
   [junit4]   2> 2723366 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723367 INFO  (zkCallback-2778-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@643fd4c7 name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723367 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723367 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue
   [junit4]   2> 2723368 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-queue-work
   [junit4]   2> 2723369 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-running
   [junit4]   2> 2723370 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-completed
   [junit4]   2> 2723371 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/collection-map-failure
   [junit4]   2> 2723372 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes
   [junit4]   2> 2723372 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /aliases.json
   [junit4]   2> 2723373 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /clusterstate.json
   [junit4]   2> 2723373 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /security.json
   [junit4]   2> 2723374 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2723374 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 2723374 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2723376 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect
   [junit4]   2> 2723376 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/election
   [junit4]   2> 2723377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2723377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96353883992948740-127.0.0.1:33120_-n_0000000000
   [junit4]   2> 2723377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33120_
   [junit4]   2> 2723377 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer_elect/leader
   [junit4]   2> 2723378 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer Overseer (id=96353883992948740-127.0.0.1:33120_-n_0000000000) starting
   [junit4]   2> 2723379 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /overseer/queue-work
   [junit4]   2> 2723381 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.OverseerAutoReplicaFailoverThread Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 2723382 INFO  (OverseerCollectionConfigSetProcessor-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
   [junit4]   2> 2723382 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer Starting to work on the main queue
   [junit4]   2> 2723382 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33120_
   [junit4]   2> 2723382 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:33120_
   [junit4]   2> 2723383 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (0)
   [junit4]   2> 2723383 INFO  (zkCallback-2778-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [0])
   [junit4]   2> 2723383 INFO  (zkCallback-2778-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2723390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2723390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2723390 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores
   [junit4]   2> 2723391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=control_collection, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2723391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1
   [junit4]   2> 2723391 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2723392 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2723392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 2723392 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2723392 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:33120_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2723397 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection control_collection not visible yet, but flagging it so a watch is registered when it becomes visible
   [junit4]   2> 2723397 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33120",
   [junit4]   2>   "node_name":"127.0.0.1:33120_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 2723398 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33120",
   [junit4]   2>   "node_name":"127.0.0.1:33120_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":null,
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2723398 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.ClusterStateMutator building a new cName: control_collection
   [junit4]   2> 2723398 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
   [junit4]   2> 2723398 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Deleting data for [control_collection]
   [junit4]   2> 2723399 INFO  (zkCallback-2778-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2723399 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController look for our core node name
   [junit4]   2> 2723399 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController waiting to find shard id in clusterstate for collection1
   [junit4]   2> 2723399 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController Check for collection zkNode:control_collection
   [junit4]   2> 2723400 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ZkController Collection zkNode exists
   [junit4]   2> 2723400 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2723401 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2723401 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1'
   [junit4]   2> 2723401 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2723401 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2723402 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.Config loaded config solrconfig.xml with version 0 
   [junit4]   2> 2723407 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2723410 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2723416 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2723419 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2723486 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 2723490 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.IndexSchema default search field in schema is text
   [junit4]   2> 2723491 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2723496 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2723497 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.FileExchangeRateProvider Reloading exchange rates from file currency.xml
   [junit4]   2> 2723498 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2723499 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.s.OpenExchangeRatesOrgProvider Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 2723500 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/configs/conf1', cwd=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 2723500 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection   x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 2723500 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2723500 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/]
   [junit4]   2> 2723500 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@13518f49
   [junit4]   2> 2723501 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/snapshot_metadata
   [junit4]   2> 2723501 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 2723501 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data
   [junit4]   2> 2723501 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/index/
   [junit4]   2> 2723501 WARN  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2723502 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/index
   [junit4]   2> 2723502 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=44, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 2723502 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@60326231 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65454343),segFN=segments_1,generation=1}
   [junit4]   2> 2723502 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2723502 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/
   [junit4]   2> 2723502 WARN  (OldIndexDirectoryCleanupThreadForCore-collection1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.DirectoryFactory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/control-001/cores/collection1/data/ does not point to a valid data directory; skipping clean-up of old index directories.
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.p.UpdateRequestProcessorChain inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 2723505 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2723505 WARN  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,args = {defaults={a=A,b=B}}}
   [junit4]   2> 2723506 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2723507 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2723507 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2723508 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2723509 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/dump,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 2723509 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2723510 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2723510 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2723510 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2723510 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2723511 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@6af49be4
   [junit4]   2> 2723511 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@60326231 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@65454343),segFN=segments_1,generation=1}
   [junit4]   2> 2723511 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2723511 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@bd7b90a[collection1] main]
   [junit4]   2> 2723511 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Load collection config from: [/collections/control_collection]
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader path=[/collections/control_collection] [configName]=[conf1] specified config exists in ZooKeeper
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {}
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 2723512 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2723513 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2723513 INFO  (searcherExecutor-5310-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@bd7b90a[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2723513 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Looking up max value of version field to seed version buckets
   [junit4]   2> 2723513 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo Refreshing highest value of _version_ for 65536 version buckets from index
   [junit4]   2> 2723513 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.VersionInfo No terms found for _version_, cannot seed version bucket highest value from index
   [junit4]   2> 2723514 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1541662144246841344
   [junit4]   2> 2723515 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.u.UpdateLog Took 1.0ms to seed version buckets with highest version 1541662144246841344
   [junit4]   2> 2723515 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController watch zkdir /configs/conf1
   [junit4]   2> 2723516 INFO  (coreLoadExecutor-5309-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2723516 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController Register replica - core:collection1 address:http://127.0.0.1:33120 collection:control_collection shard:shard1
   [junit4]   2> 2723517 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 2723519 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/control_collection/leader_elect/shard1/election/96353883992948740-core_node1-n_0000000000
   [junit4]   2> 2723520 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
   [junit4]   2> 2723521 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2723521 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2723521 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33120/collection1/
   [junit4]   2> 2723521 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 2723521 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection"} current state version: 1
   [junit4]   2> 2723522 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:33120/collection1/ has no replicas
   [junit4]   2> 2723522 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.SolrZkClient makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 2723524 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/96353883992948740-core_node1-n_0000000000
   [junit4]   2> 2723526 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33120/collection1/ shard1
   [junit4]   2> 2723526 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "base_url":"http://127.0.0.1:33120",
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "state":"active"} current state version: 1
   [junit4]   2> 2723628 INFO  (zkCallback-2778-thread-1-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2723677 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController We are http://127.0.0.1:33120/collection1/ and leader is http://127.0.0.1:33120/collection1/
   [junit4]   2> 2723677 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33120
   [junit4]   2> 2723677 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2723677 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController publishing state=active
   [junit4]   2> 2723677 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2723678 INFO  (coreZkRegister-5303-thread-1-processing-n:127.0.0.1:33120_ x:collection1 s:shard1 c:control_collection r:core_node1) [n:127.0.0.1:33120_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.c.ZkStateReader Checking legacy cluster state for collection control_collection
   [junit4]   2> 2723678 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33120",
   [junit4]   2>   "node_name":"127.0.0.1:33120_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"} current state version: 2
   [junit4]   2> 2723678 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "core_node_name":"core_node1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:33120",
   [junit4]   2>   "node_name":"127.0.0.1:33120_",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"control_collection",
   [junit4]   2>   "operation":"state"}
   [junit4]   2> 2723780 INFO  (zkCallback-2778-thread-3-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2723893 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723893 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723894 INFO  (zkCallback-2782-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@54e169f name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2723894 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 2723895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2723895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 2723895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
   [junit4]   2> 2723895 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723896 INFO  (zkCallback-2783-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@33aee513 name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723896 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723901 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"create",
   [junit4]   2>   "name":"collection1",
   [junit4]   2>   "numShards":"2",
   [junit4]   2>   "stateFormat":"2"} current state version: 3
   [junit4]   2> 2723901 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.ClusterStateMutator building a new cName: collection1
   [junit4]   2> 2723956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/cores/collection1
   [junit4]   2> 2723956 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001
   [junit4]   2> 2723957 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2723958 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@77da2d29{/,null,AVAILABLE}
   [junit4]   2> 2723958 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.ServerConnector Started ServerConnector@23f27303{HTTP/1.1,[http/1.1]}{127.0.0.1:46685}
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.e.j.s.Server Started @2726183ms
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=46685, coreRootDirectory=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/cores}
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001'
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2723959 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723960 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723960 INFO  (zkCallback-2784-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@320d42b2 name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723960 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723961 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723961 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 2723961 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/solr.xml
   [junit4]   2> 2723964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/cores
   [junit4]   2> 2723964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CoreContainer New CoreContainer 1656449871
   [junit4]   2> 2723964 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001]
   [junit4]   2> 2723964 WARN  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/lib
   [junit4]   2> 2723965 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2723965 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2723966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2723966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2723966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50426/solr
   [junit4]   2> 2723966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.ZkController zkHost includes chroot
   [junit4]   2> 2723966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkCredentialsProvider
   [junit4]   2> 2723966 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723967 INFO  (zkCallback-2787-thread-1) [    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@5504ee25 name:ZooKeeperConnection Watcher:127.0.0.1:50426 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [    ] o.a.s.c.c.SolrZkClient Using default ZkACLProvider
   [junit4]   2> 2723968 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$12/976932739@2777a411
   [junit4]   2> 2723969 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 2723969 INFO  (zkCallback-2788-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_    ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@434ba52e name:ZooKeeperConnection Watcher:127.0.0.1:50426/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 2723969 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 2723972 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.c.ZkStateReader Updating cluster state from ZooKeeper... 
   [junit4]   2> 2723972 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.c.ZkStateReader Loaded empty cluster properties
   [junit4]   2> 2723973 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2723974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 2723974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/96353883992948745-127.0.0.1:46685_-n_0000000001
   [junit4]   2> 2723974 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/96353883992948740-127.0.0.1:33120_-n_0000000000 to know if I could be the leader
   [junit4]   2> 2723975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.ZkController Publish node=127.0.0.1:46685_ as DOWN
   [junit4]   2> 2723975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46685_
   [junit4]   2> 2723975 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.c.SolrZkClient makePath: /live_nodes/127.0.0.1:46685_
   [junit4]   2> 2723975 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:46685_"} current state version: 3
   [junit4]   2> 2723975 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:46685_
   [junit4]   2> 2723976 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/collection1/state.json
   [junit4]   2> 2723976 INFO  (zkCallback-2782-thread-1) [    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2723976 INFO  (zkCallback-2778-thread-2-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2723976 INFO  (zkCallback-2782-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2723976 INFO  (zkCallback-2788-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_    ] o.a.s.c.c.ZkStateReader A live node change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/live_nodes], has occurred - updating... (live nodes size: [1])
   [junit4]   2> 2723976 INFO  (zkCallback-2778-thread-2-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2723976 INFO  (zkCallback-2788-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2723977 INFO  (zkCallback-2778-thread-2-processing-n:127.0.0.1:33120_) [n:127.0.0.1:33120_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2723977 INFO  (zkCallback-2782-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2723977 INFO  (zkCallback-2788-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json], has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2723982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2723982 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2723983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/cores
   [junit4]   2> 2723983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, transient=false, schema=schema.xml, loadOnStartup=true, configSetProperties=configsetprops.json, collection=collection1, coreNodeName=, dataDir=data/, shard=}
   [junit4]   2> 2723983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/../../../../../../../../../../x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2/temp/solr.cloud.CollectionsAPIDistributedZkTest_146D5B52C5AF51A-001/shard-1-001/cores/collection1
   [junit4]   2> 2723983 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2723984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J2
   [junit4]   2> 2723984 INFO  (TEST-CollectionsAPIDistributedZkTest.test-seed#[146D5B52C5AF51A]) [n:127.0.0.1:46685_    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2723984 INFO  (coreLoadExecutor-5320-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_ c:collection1   x:collection1] o.a.s.c.ZkController publishing state=down
   [junit4]   2> 2723984 INFO  (coreLoadExecutor-5320-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_ c:collection1   x:collection1] o.a.s.c.ZkController numShards not found on descriptor - reading it from system property
   [junit4]   2> 2723985 INFO  (coreLoadExecutor-5320-thread-1-processing-n:127.0.0.1:46685_) [n:127.0.0.1:46685_ c:collection1   x:collection1] o.a.s.c.ZkController Registering watch for collection collection1
   [junit4]   2> 2723985 INFO  (OverseerStateUpdate-96353883992948740-127.0.0.1:33120_-n_0000000000) [n:127.0.0.1:33120_    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection1",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:46685",
   [junit4]   2>   "node_name":"127.0.0.1:46685_",
   [junit4]   2>   

[...truncated too long message...]

exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160803_161425_766.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160803_161425_766.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 264 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 134 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/134/

3 tests failed.
FAILED:  org.apache.solr.cloud.TestRandomRequestDistribution.test

Error Message:
Shard a1x2_shard1_replica1 received all 10 requests

Stack Trace:
java.lang.AssertionError: Shard a1x2_shard1_replica1 received all 10 requests
	at __randomizedtesting.SeedInfo.seed([146FB096C07DA498:9C3B8F4C6E81C960]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.cloud.TestRandomRequestDistribution.testRequestTracking(TestRandomRequestDistribution.java:121)
	at org.apache.solr.cloud.TestRandomRequestDistribution.test(TestRandomRequestDistribution.java:65)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 10 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 10 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([146FB096C07DA498]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.solr.core.TestDynamicLoading.testDynamicLoading

Error Message:
Could not get expected value  'X val' for path 'x' full output: {   "responseHeader":{     "status":0,     "QTime":0},   "params":{"wt":"json"},   "context":{     "webapp":"/xgjyk",     "path":"/test1",     "httpMethod":"GET"},   "class":"org.apache.solr.core.BlobStoreTestRequestHandler",   "x":null},  from server:  null

Stack Trace:
java.lang.AssertionError: Could not get expected value  'X val' for path 'x' full output: {
  "responseHeader":{
    "status":0,
    "QTime":0},
  "params":{"wt":"json"},
  "context":{
    "webapp":"/xgjyk",
    "path":"/test1",
    "httpMethod":"GET"},
  "class":"org.apache.solr.core.BlobStoreTestRequestHandler",
  "x":null},  from server:  null
	at __randomizedtesting.SeedInfo.seed([146FB096C07DA498:CC229DC137A00138]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:481)
	at org.apache.solr.core.TestDynamicLoading.testDynamicLoading(TestDynamicLoading.java:232)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 11981 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandler
   [junit4]   2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/init-core-data-001
   [junit4]   2> 2025312 INFO  (SUITE-TestReplicationHandler-seed#[146FB096C07DA498]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4]   2> 2025313 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.SolrTestCaseJ4 ###Starting doTestReplicateAfterStartupWithNoActivity
   [junit4]   2> 2025314 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1
   [junit4]   2> 2025322 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2025323 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b35cd35{/solr,null,AVAILABLE}
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.ServerConnector Started ServerConnector@1c6edbed{HTTP/1.1,[http/1.1]}{127.0.0.1:43550}
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.Server Started @2027626ms
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=43550}
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001'
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2025325 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/solr.xml
   [junit4]   2> 2025329 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/.
   [junit4]   2> 2025329 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer New CoreContainer 2059679889
   [junit4]   2> 2025329 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001]
   [junit4]   2> 2025329 WARN  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/lib
   [junit4]   2> 2025329 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2025330 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2025330 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2025330 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2025331 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2025331 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2025331 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/.
   [junit4]   2> 2025332 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 2025332 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1
   [junit4]   2> 2025332 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2025332 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1'
   [junit4]   2> 2025332 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 2025333 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2025333 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2025333 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2025339 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2025341 WARN  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2025342 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2025346 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2025348 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2025350 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2025351 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1'
   [junit4]   2> 2025351 INFO  (coreLoadExecutor-4278-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1
   [junit4]   2> 2025351 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2025351 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 2025351 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/snapshot_metadata
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 2025352 WARN  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=65.857421875, floorSegmentMB=2.123046875, forceMergeDeletesPctAllowed=10.510085272075946, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.706243987464695
   [junit4]   2> 2025352 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4defe29e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@521f464f),segFN=segments_1,generation=1}
   [junit4]   2> 2025353 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2025353 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/
   [junit4]   2> 2025354 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2025355 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025355 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025356 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025357 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025357 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2025357 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,lazy,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 2025357 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=211309566, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4defe29e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@521f464f),segFN=segments_1,generation=1}
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6b8ecd57[collection1] main]
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/conf
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/conf}
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/conf
   [junit4]   2> 2025358 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2025359 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
   [junit4]   2> 2025359 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2025359 INFO  (searcherExecutor-4279-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@6b8ecd57[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2025359 INFO  (coreLoadExecutor-4278-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2025834 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1
   [junit4]   2> 2025838 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2025839 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@37f99303{/solr,null,AVAILABLE}
   [junit4]   2> 2025839 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.ServerConnector Started ServerConnector@2313dc1c{HTTP/1.1,[http/1.1]}{127.0.0.1:34886}
   [junit4]   2> 2025839 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.Server Started @2028141ms
   [junit4]   2> 2025840 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=34886}
   [junit4]   2> 2025840 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2025840 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002'
   [junit4]   2> 2025842 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2025842 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2025844 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/solr.xml
   [junit4]   2> 2025847 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/.
   [junit4]   2> 2025848 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer New CoreContainer 502734745
   [junit4]   2> 2025848 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002]
   [junit4]   2> 2025848 WARN  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/lib
   [junit4]   2> 2025848 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2025849 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2025849 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2025849 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2025849 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2025849 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2025850 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/.
   [junit4]   2> 2025850 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 2025850 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1
   [junit4]   2> 2025850 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2025853 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1'
   [junit4]   2> 2025855 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 2025855 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2025855 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2025855 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2025858 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2025860 WARN  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2025861 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2025865 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2025867 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2025868 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2025869 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1'
   [junit4]   2> 2025869 INFO  (coreLoadExecutor-4288-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1
   [junit4]   2> 2025869 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 2025869 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/]
   [junit4]   2> 2025869 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/snapshot_metadata
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/index/
   [junit4]   2> 2025870 WARN  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=50, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=65.857421875, floorSegmentMB=2.123046875, forceMergeDeletesPctAllowed=10.510085272075946, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.706243987464695
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@c7fcbce lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@29feffae),segFN=segments_1,generation=1}
   [junit4]   2> 2025870 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2025871 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/
   [junit4]   2> 2025872 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2025872 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025873 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025873 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025874 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2025874 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,lazy,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=211309566, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@c7fcbce lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@29feffae),segFN=segments_1,generation=1}
   [junit4]   2> 2025875 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2025876 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@386f0884[collection1] main]
   [junit4]   2> 2025876 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1/conf
   [junit4]   2> 2025876 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1/conf}
   [junit4]   2> 2025876 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/collection1/conf
   [junit4]   2> 2025876 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2025877 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 2025877 INFO  (searcherExecutor-4289-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@386f0884[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2025877 INFO  (coreLoadExecutor-4288-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2026356 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@2313dc1c{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2026356 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=502734745
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@388fe234
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0}
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 3 directories currently being tracked
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data;done=false>>]
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/snapshot_metadata [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/snapshot_metadata;done=false>>]
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/snapshot_metadata
   [junit4]   2> 2026357 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/index;done=false>>]
   [junit4]   2> 2026358 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2026358 INFO  (coreCloseExecutor-4294-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-002/./collection1/data/index
   [junit4]   2> 2026358 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@37f99303{/solr,null,UNAVAILABLE}
   [junit4]   2> 2026361 INFO  (qtp221511252-59611) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 2026361 INFO  (qtp221511252-59611) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
   [junit4]   2> 2026363 INFO  (qtp221511252-59612) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2026364 INFO  (qtp221511252-59612) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4defe29e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@521f464f),segFN=segments_1,generation=1}
   [junit4]   2> 	commit{dir=MockDirectoryWrapper(RAMDirectory@4defe29e lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@521f464f),segFN=segments_2,generation=2}
   [junit4]   2> 2026365 INFO  (qtp221511252-59612) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 2
   [junit4]   2> 2026367 INFO  (qtp221511252-59612) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@34acf75b[collection1] main]
   [junit4]   2> 2026367 INFO  (qtp221511252-59612) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2026367 INFO  (searcherExecutor-4279-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@34acf75b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2026367 INFO  (qtp221511252-59612) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 4
   [junit4]   2> 2026369 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.ServerConnector Stopped ServerConnector@1c6edbed{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 2026369 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2059679889
   [junit4]   2> 2026371 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@733c287b
   [junit4]   2> 2026371 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.u.DirectUpdateHandler2 closing DirectUpdateHandler2{commits=1,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=1,cumulative_errors=0}
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.u.SolrCoreState Closing SolrCoreState
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.u.DefaultSolrCoreState closing IndexWriter with IndexWriterCloser
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Closing main searcher on request.
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing MockDirectoryFactory - 3 directories currently being tracked
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data;done=false>>]
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index;done=false>>]
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory looking to close /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/snapshot_metadata [CachedDir<<refCount=0;path=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/snapshot_metadata;done=false>>]
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory, CoreContainer#isShutdown=true
   [junit4]   2> 2026372 INFO  (coreCloseExecutor-4296-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory Closing directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/snapshot_metadata
   [junit4]   2> 2026373 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2b35cd35{/solr,null,UNAVAILABLE}
   [junit4]   2> 2026374 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.Server jetty-9.3.8.v20160314
   [junit4]   2> 2026378 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@9985892{/solr,null,AVAILABLE}
   [junit4]   2> 2026378 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.ServerConnector Started ServerConnector@42be38f8{HTTP/1.1,[http/1.1]}{127.0.0.1:57422}
   [junit4]   2> 2026378 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.e.j.s.Server Started @2028680ms
   [junit4]   2> 2026378 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=57422}
   [junit4]   2> 2026379 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init(): sun.misc.Launcher$AppClassLoader@73d16e93
   [junit4]   2> 2026379 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001'
   [junit4]   2> 2026379 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2026379 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2026379 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/solr.xml
   [junit4]   2> 2026382 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Config-defined core root directory: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/.
   [junit4]   2> 2026382 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer New CoreContainer 1145471155
   [junit4]   2> 2026382 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Loading cores into CoreContainer [instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001]
   [junit4]   2> 2026382 WARN  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/lib to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/lib
   [junit4]   2> 2026383 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.h.c.HttpShardHandlerFactory created with socketTimeout : 90000,urlScheme : ,connTimeout : 15000,maxConnectionsPerHost : 20,maxConnections : 10000,corePoolSize : 0,maximumPoolSize : 2147483647,maxThreadIdleTime : 5,sizeOfQueue : -1,fairnessPolicy : false,useRetries : false,connectionsEvictorSleepDelay : 5000,maxConnectionIdleTime : 40000,
   [junit4]   2> 2026386 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
   [junit4]   2> 2026386 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.l.LogWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 2026386 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.l.LogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 2026386 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer Security conf doesn't exist. Skipping setup for authorization module.
   [junit4]   2> 2026386 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreContainer No authentication plugin used.
   [junit4]   2> 2026386 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Looking for core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/.
   [junit4]   2> 2026387 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CoreDescriptor Created CoreDescriptor: {name=collection1, config=solrconfig.xml, loadOnStartup=true, schema=schema.xml, configSetProperties=configsetprops.json, transient=false, dataDir=data/}
   [junit4]   2> 2026387 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Found core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1
   [junit4]   2> 2026387 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions
   [junit4]   2> 2026387 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.SolrResourceLoader new SolrResourceLoader for directory: '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1'
   [junit4]   2> 2026395 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.SolrResourceLoader JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 2026395 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
   [junit4]   2> 2026395 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter user.dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0
   [junit4]   2> 2026395 INFO  (TEST-TestReplicationHandler.doTestReplicateAfterStartupWithNoActivity-seed#[146FB096C07DA498]) [    ] o.a.s.s.SolrDispatchFilter SolrDispatchFilter.init() done
   [junit4]   2> 2026398 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.SolrConfig current version of requestparams : -1
   [junit4]   2> 2026400 WARN  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
   [junit4]   2> 2026400 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.2.0
   [junit4]   2> 2026405 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.SolrConfig Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 2026406 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 2026408 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.s.IndexSchema unique key field: id
   [junit4]   2> 2026408 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.ConfigSetProperties Did not find ConfigSet properties, assuming default properties: Can't find resource 'configsetprops.json' in classpath or '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1'
   [junit4]   2> 2026408 INFO  (coreLoadExecutor-4303-thread-1) [    ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1
   [junit4]   2> 2026409 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.NRTCachingDirectoryFactory
   [junit4]   2> 2026413 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/]
   [junit4]   2> 2026413 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore JMX monitoring not detected for core: collection1
   [junit4]   2> 2026413 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data/snapshot_metadata
   [junit4]   2> 2026413 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.s.SolrSnapshotMetaDataManager Loading from snapshot metadata file...
   [junit4]   2> 2026413 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data
   [junit4]   2> 2026413 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore New index directory detected: old=null new=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index/
   [junit4]   2> 2026413 WARN  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1] Solr index directory '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/index' doesn't exist. Creating new index...
   [junit4]   2> 2026414 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.CachingDirectoryFactory return new directory for /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data/index
   [junit4]   2> 2026414 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=5, maxMergeAtOnceExplicit=9, maxMergedSegmentMB=39.6455078125, floorSegmentMB=1.498046875, forceMergeDeletesPctAllowed=5.661925787057958, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4052861862136866
   [junit4]   2> 2026418 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@688343a8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
   [junit4]   2> 2026418 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2026418 INFO  (OldIndexDirectoryCleanupThreadForCore-collection1) [    x:collection1] o.a.s.c.SolrCore Looking for old index directories to cleanup for core collection1 in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/./collection1/data/
   [junit4]   2> 2026419 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 2026420 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2026420 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2026421 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2026421 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.h.l.XMLLoader xsltCacheLifetimeSeconds=60
   [junit4]   2> 2026422 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.PluginBag requestHandler : 'lazy' created with startup=lazy 
   [junit4]   2> 2026422 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.RequestHandlers Registered paths: /admin/mbeans,standard,lazy,/update/json/docs,/admin/luke,/export,/get,/admin/properties,/update/json,/admin/threads,/update/csv,/sql,/graph,/admin/segments,/admin/system,/replication,defaults,/config,/stream,/schema,/admin/plugins,/admin/logging,/admin/ping,/update,/admin/file,/terms
   [junit4]   2> 2026422 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrCore Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 2026422 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2026422 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2026422 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=40, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.2181944154849793]
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 	commit{dir=NRTCachingDirectory(MMapDirectory@/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/data/index lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@688343a8; maxCacheMB=48.0 maxMergeSizeMB=4.0),segFN=segments_1,generation=1}
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.SolrDeletionPolicy newest commit generation = 1
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@30a979dd[collection1] main]
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/conf
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing RestManager with initArgs: {storageDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/conf}
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Reading _rest_managed.json using file:dir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_146FB096C07DA498-001/solr-instance-001/collection1/conf
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.r.RestManager Initializing 0 registered ManagedResources
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for  10000
   [junit4]   2> 2026423 INFO  (searcherExecutor-4304-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@30a979dd[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2026423 INFO  (coreLoadExecutor-4303-thread-1) [    x:collection1] o.a.s.c.CoreContainer registering core: collection1
   [junit4]   2> 2026898 INFO  (qtp1268612900-59653) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 0
   [junit4]   2> 2026900 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
   [junit4]   2> 2026902 INFO  (qtp1268612900-59655) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
   [junit4]   2> 2026904 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
   [junit4]   2> 2026907 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
   [junit4]   2> 2026909 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
   [junit4]   2> 2026910 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
   [junit4]   2> 2026912 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
   [junit4]   2> 2026914 INFO  (qtp1268612900-59655) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
   [junit4]   2> 2026915 INFO  (qtp1268612900-59655) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
   [junit4]   2> 2026920 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
   [junit4]   2> 2026922 INFO  (qtp1268612900-59653) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
   [junit4]   2> 2026923 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
   [junit4]   2> 2026925 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
   [junit4]   2> 2026927 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
   [junit4]   2> 2026928 INFO  (qtp1268612900-59655) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
   [junit4]   2> 2026930 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
   [junit4]   2> 2026932 INFO  (qtp1268612900-59653) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
   [junit4]   2> 2026933 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
   [junit4]   2> 2026935 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
   [junit4]   2> 2026936 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
   [junit4]   2> 2026938 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
   [junit4]   2> 2026940 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
   [junit4]   2> 2026941 INFO  (qtp1268612900-59653) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
   [junit4]   2> 2026943 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
   [junit4]   2> 2026944 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
   [junit4]   2> 2026946 INFO  (qtp1268612900-59655) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
   [junit4]   2> 2026947 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
   [junit4]   2> 2026949 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
   [junit4]   2> 2026951 INFO  (qtp1268612900-59653) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
   [junit4]   2> 2026953 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
   [junit4]   2> 2026955 INFO  (qtp1268612900-59650) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
   [junit4]   2> 2026956 INFO  (qtp1268612900-59655) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
   [junit4]   2> 2026958 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
   [junit4]   2> 2026961 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
   [junit4]   2> 2026962 INFO  (qtp1268612900-59657) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
   [junit4]   2> 2026964 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
   [junit4]   2> 2026966 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
   [junit4]   2> 2026967 INFO  (qtp1268612900-59654) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
   [junit4]   2> 2026969 INFO  (qtp1268612900-59656) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
   [junit4

[...truncated too long message...]

exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160802_174303_058.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J2-20160802_174303_058.suites -stdin
   [junit4] ERROR: JVM J2 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 615 minutes 40 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any




[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 133 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/133/

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsChaosMonkeySafeLeaderTest

Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]
	at __randomizedtesting.SeedInfo.seed([798AF5306665F37A]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.TestAtomicUpdateErrorCases

Error Message:


Stack Trace:
java.lang.AssertionError
	at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.before(TestRuleTemporaryFilesCleanup.java:112)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.before(TestRuleAdapter.java:26)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:35)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.update.TestAtomicUpdateErrorCases

Error Message:
176 threads leaked from SUITE scope at org.apache.solr.update.TestAtomicUpdateErrorCases:     1) Thread[id=56032, name=coreZkRegister-2064-thread-11, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=56050, name=coreZkRegister-2257-thread-13, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=56110, name=watches-741-thread-24, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=56046, name=coreZkRegister-2257-thread-9, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=56219, name=qtp1537330087-56219, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)         at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)         at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)         at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:575)         at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:93)         at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:61)         at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1646)         at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1623)         at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:160)         at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:68)         at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:62)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:518)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=55982, name=zkCallback-740-thread-27, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=56081, name=watches-722-thread-14, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=56103, name=watches-741-thread-23, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=56016, name=coreZkRegister-2053-thread-3, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=56229, name=qtp1537330087-56229, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)         at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)         at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)         at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:575)         at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:93)         at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:61)         at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1646)         at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1623)         at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:160)         at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:68)         at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:62)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:518)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=56014, name=OverseerCollectionConfigSetProcessor-96341220247732282-127.0.0.1:54687_-n_0000000006, state=TIMED_WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.apache.solr.cloud.DistributedQueue.peekElements(DistributedQueue.java:343)         at org.apache.solr.cloud.OverseerTaskQueue.peekTopN(OverseerTaskQueue.java:235)         at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:224)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=56040, name=coreZkRegister-2031-thread-7, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=55997, name=zkCallback-740-thread-33-processing-n:127.0.0.1:59829_-EventThread, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)         at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)         at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:494)   14) Thread[id=55993, name=zkCallback-721-thread-19-processing-n:127.0.0.1:41992_-SendThread(127.0.0.1:45397), state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:349)         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)   15) Thread[id=56101, name=watches-781-thread-16, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   16) Thread[id=56217, name=qtp1537330087-56217, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.hadoop.ipc.Client.call(Client.java:1466)         at org.apache.hadoop.ipc.Client.call(Client.java:1412)         at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)         at com.sun.proxy.$Proxy51.getFileInfo(Unknown Source)         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)         at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)         at java.lang.reflect.Method.invoke(Method.java:498)         at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)         at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)         at com.sun.proxy.$Proxy55.getFileInfo(Unknown Source)         at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108)         at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)         at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)         at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)         at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1317)         at org.apache.solr.store.hdfs.HdfsDirectory$HdfsIndexInput.<init>(HdfsDirectory.java:212)         at org.apache.solr.store.hdfs.HdfsDirectory.openInput(HdfsDirectory.java:139)         at org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:192)         at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:327)         at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1577)         at org.apache.solr.update.UpdateLog.openRealtimeSearcher(UpdateLog.java:517)         at org.apache.solr.update.IndexFingerprint.getFingerprint(IndexFingerprint.java:83)         at org.apache.solr.handler.component.RealTimeGetComponent.processGetVersions(RealTimeGetComponent.java:626)         at org.apache.solr.handler.component.RealTimeGetComponent.process(RealTimeGetComponent.java:117)         at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:518)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   17) Thread[id=55972, name=zkCallback-747-thread-8, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   18) Thread[id=55989, name=zkCallback-740-thread-34, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   19) Thread[id=56117, name=recoveryExecutor-778-thread-13-processing-n:127.0.0.1:54687_ x:collection1 s:shard1 c:collection1 r:core_node2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.net.SocketInputStream.socketRead0(Native Method)         at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)         at java.net.SocketInputStream.read(SocketInputStream.java:170)         at java.net.SocketInputStream.read(SocketInputStream.java:141)         at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)         at sun.security.ssl.InputRecord.read(InputRecord.java:503)         at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)         at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)         at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)         at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)         at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)         at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)         at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)         at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)         at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)         at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)         at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)         at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)         at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)         at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)         at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)         at org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:201)         at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:141)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:408)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:221)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$31/1587311309.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   20) Thread[id=55978, name=zkCallback-740-thread-23, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   21) Thread[id=56049, name=coreZkRegister-2257-thread-12, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   22) Thread[id=56090, name=watches-741-thread-22, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   23) Thread[id=56026, name=coreZkRegister-2064-thread-5, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   24) Thread[id=55990, name=zkCallback-780-thread-30, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   25) Thread[id=56067, name=watches-741-thread-20, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   26) Thread[id=56070, name=watches-734-thread-13, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   27) Thread[id=55986, name=zkCallback-740-thread-31, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   28) Thread[id=56073, name=coreZkRegister-2053-thread-7, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   29) Thread[id=56004, name=watches-734-thread-11, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   30) Thread[id=56105, name=watches-722-thread-18, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   31) Thread[id=56021, name=OverseerThreadFactory-2837-thread-5, state=WAITING, group=Overseer collection creation process.]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   32) Thread[id=55985, name=zkCallback-740-thread-30, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   33) Thread[id=55965, name=zkCallback-780-thread-24, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   34) Thread[id=56005, name=watches-741-thread-13, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   35) Thread[id=55974, name=zkCallback-747-thread-10, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   36) Thread[id=56212, name=qtp2052259692-56212, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)         at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)         at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)         at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:575)         at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:93)         at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:61)         at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1646)         at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1623)         at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:160)         at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:68)         at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:62)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:518)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   37) Thread[id=56211, name=qtp2052259692-56211, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.lang.Object.wait(Native Method)         at java.lang.Object.wait(Object.java:502)         at org.apache.hadoop.ipc.Client.call(Client.java:1466)         at org.apache.hadoop.ipc.Client.call(Client.java:1412)         at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)         at com.sun.proxy.$Proxy51.getFileInfo(Unknown Source)         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)         at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source)         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)         at java.lang.reflect.Method.invoke(Method.java:498)         at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)         at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)         at com.sun.proxy.$Proxy55.getFileInfo(Unknown Source)         at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108)         at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)         at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)         at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)         at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1317)         at org.apache.solr.store.hdfs.HdfsDirectory$HdfsIndexInput.<init>(HdfsDirectory.java:212)         at org.apache.solr.store.hdfs.HdfsDirectory.openInput(HdfsDirectory.java:139)         at org.apache.lucene.store.NRTCachingDirectory.openInput(NRTCachingDirectory.java:192)         at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:327)         at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1577)         at org.apache.solr.update.UpdateLog.openRealtimeSearcher(UpdateLog.java:517)         at org.apache.solr.update.IndexFingerprint.getFingerprint(IndexFingerprint.java:83)         at org.apache.solr.handler.component.RealTimeGetComponent.processGetVersions(RealTimeGetComponent.java:626)         at org.apache.solr.handler.component.RealTimeGetComponent.process(RealTimeGetComponent.java:117)         at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:295)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:518)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   38) Thread[id=56114, name=updateExecutor-777-thread-13-processing-n:127.0.0.1:54687_ x:collection1 s:shard1 c:collection1 r:core_node2, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)         at java.util.concurrent.FutureTask.get(FutureTask.java:191)         at org.apache.solr.update.DefaultSolrCoreState$2.run(DefaultSolrCoreState.java:317)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$31/1587311309.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   39) Thread[id=56034, name=zkCallback-733-thread-33, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   40) Thread[id=56054, name=coreZkRegister-2053-thread-4, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   41) Thread[id=56215, name=qtp1537330087-56215, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)         at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)         at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)         at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)         at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:575)         at org.apache.solr.update.processor.RunUpdateProcessor.processCommit(RunUpdateProcessorFactory.java:93)         at org.apache.solr.update.processor.UpdateRequestProcessor.processCommit(UpdateRequestProcessor.java:61)         at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalCommit(DistributedUpdateProcessor.java:1646)         at org.apache.solr.update.processor.DistributedUpdateProcessor.processCommit(DistributedUpdateProcessor.java:1623)         at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processCommit(LogUpdateProcessorFactory.java:160)         at org.apache.solr.handler.RequestHandlerUtils.handleCommit(RequestHandlerUtils.java:68)         at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:62)         at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:154)         at org.apache.solr.core.SolrCore.execute(SolrCore.java:2089)         at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)         at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:459)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)         at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:108)         at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)         at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)         at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)         at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)         at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)         at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)         at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)         at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)         at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:399)         at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)         at org.eclipse.jetty.server.Server.handle(Server.java:518)         at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)         at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:186)         at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)         at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)         at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   42) Thread[id=56015, name=OverseerHdfsCoreFailoverThread-96341220247732282-127.0.0.1:54687_-n_0000000006, state=TIMED_WAITING, group=Overseer Hdfs SolrCore Failover Thread.]         at java.lang.Thread.sleep(Native Method)         at org.apache.solr.cloud.OverseerAutoReplicaFailoverThread.run(OverseerAutoReplicaFailoverThread.java:137)         at java.lang.Thread.run(Thread.java:745)   43) Thread[id=55995, name=IPC Client (1543981367) connection to localhost/127.0.0.1:44470 from jenkins, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335)         at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)         at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161)         at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131)         at java.io.FilterInputStream.read(FilterInputStream.java:133)         at java.io.FilterInputStream.read(FilterInputStream.java:133)         at org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:520)         at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)         at java.io.BufferedInputStream.read(BufferedInputStream.java:265)         at java.io.DataInputStream.readInt(DataInputStream.java:387)         at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1084)         at org.apache.hadoop.ipc.Client$Connection.run(Client.java:979)   44) Thread[id=56042, name=coreZkRegister-2031-thread-9, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   45) Thread[id=56142, name=recoveryExecutor-719-thread-8-processing-n:127.0.0.1:41992_ x:awholynewcollection_1_shard2_replica1 s:shard2 c:awholynewcollection_1 r:core_node9, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.net.SocketInputStream.socketRead0(Native Method)         at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)         at java.net.SocketInputStream.read(SocketInputStream.java:170)         at java.net.SocketInputStream.read(SocketInputStream.java:141)         at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)         at sun.security.ssl.InputRecord.read(InputRecord.java:503)         at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)         at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)         at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)         at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)         at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)         at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)         at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)         at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)         at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)         at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)         at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)         at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)         at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)         at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)         at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)         at org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:201)         at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:141)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:408)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:221)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$31/1587311309.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   46) Thread[id=56033, name=coreZkRegister-2064-thread-12, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   47) Thread[id=56148, name=watches-734-thread-20, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   48) Thread[id=56189, name=recoveryExecutor-738-thread-5-processing-n:127.0.0.1:59829_ x:awholynewcollection_1_shard3_replica3 s:shard3 c:awholynewcollection_1 r:core_node2, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at java.net.SocketInputStream.socketRead0(Native Method)         at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)         at java.net.SocketInputStream.read(SocketInputStream.java:170)         at java.net.SocketInputStream.read(SocketInputStream.java:141)         at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)         at sun.security.ssl.InputRecord.read(InputRecord.java:503)         at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)         at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)         at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)         at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)         at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)         at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)         at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)         at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)         at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)         at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)         at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)         at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)         at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)         at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)         at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)         at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)         at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)         at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:149)         at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:166)         at org.apache.solr.cloud.RecoveryStrategy.commitOnLeader(RecoveryStrategy.java:201)         at org.apache.solr.cloud.RecoveryStrategy.replicate(RecoveryStrategy.java:141)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:408)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:221)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$31/1587311309.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   49) Thread[id=56149, name=watches-722-thread-21, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   50) Thread[id=56235, name=qtp2052259692-56235, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:546)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:47)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:609)         at java.lang.Thread.run(Thread.java:745)   51) Thread[id=56023, name=coreZkRegister-2064-thread-2, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   52) Thread[id=55964, name=zkCallback-780-thread-23, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   53) Thread[id=56083, name=watches-722-thread-15, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   54) Thread[id=56025, name=coreZkRegister-2064-thread-4, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   55) Thread[id=56146, name=recoveryExecutor-731-thread-8-processing-n:127.0.0.1:41170_ x:awholynewstresscollection_collection1_0_shard4_replica1 s:shard4 c:awholynewstresscollection_collection1_0 r:core_node7, state=BLOCKED, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at org.apache.solr.update.UpdateLog.getRecentUpdates(UpdateLog.java:1082)         at org.apache.solr.cloud.RecoveryStrategy.doRecovery(RecoveryStrategy.java:252)         at org.apache.solr.cloud.RecoveryStrategy.run(RecoveryStrategy.java:221)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$31/1587311309.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   56) Thread[id=56011, name=zkCallback-733-thread-32, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   57) Thread[id=56193, name=updateExecutor-730-thread-11-processing-n:127.0.0.1:41170_ x:awholynewcollection_2_shard1_replica4 s:shard1 c:awholynewcollection_2 r:core_node3, state=WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)         at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:429)         at java.util.concurrent.FutureTask.get(FutureTask.java:191)         at org.apache.solr.update.DefaultSolrCoreState$2.run(DefaultSolrCoreState.java:317)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.run(FutureTask.java:266)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)         at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$31/1587311309.run(Unknown Source)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   58) Thread[id=56095, name=coreZkRegister-2053-thread-11, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   59) Thread[id=56024, name=coreZkRegister-2064-thread-3, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   60) Thread[id=56001, name=qtp1586931699-56001, state=RUNNABLE, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)         at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)         at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93)         at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)         at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:233)         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:171)         at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)   61) Thread[id=56039, name=coreZkRegister-2031-thread-6, state=TIMED_WAITING, group=TGRP-HdfsCollectionsAPIDistributedZkTest]         at sun.misc.Unsafe.park(Native Method)   

[...truncated too long message...]

e-lucene-core:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


compile-codecs:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


common.init:


compile-lucene-core:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


common.compile-core:


compile-core:


compile-core:


compile-solr-core:


compile-solrj:


check-lucene-core-uptodate:


jar-lucene-core:


check-backward-codecs-uptodate:


jar-backward-codecs:


check-analyzers-phonetic-uptodate:


jar-analyzers-phonetic:


check-analyzers-kuromoji-uptodate:


jar-analyzers-kuromoji:


check-codecs-uptodate:


jar-codecs:


check-expressions-uptodate:


jar-expressions:


check-suggest-uptodate:


jar-suggest:


check-highlighter-uptodate:


jar-highlighter:


check-memory-uptodate:


jar-memory:


check-misc-uptodate:


jar-misc:


check-spatial-extras-uptodate:


jar-spatial-extras:


check-grouping-uptodate:


jar-grouping:


check-queries-uptodate:


jar-queries:


check-queryparser-uptodate:


jar-queryparser:


check-join-uptodate:


jar-join:


check-sandbox-uptodate:


jar-sandbox:


check-classification-uptodate:


jar-classification:


prep-lucene-jars:


resolve-example:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


resolve-server:


ivy-availability-check:


ivy-fail:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


common.init:


compile-lucene-core:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


common.compile-core:


compile-core:


check-lucene-core-uptodate:


jar-lucene-core:


check-backward-codecs-uptodate:


jar-backward-codecs:


check-analyzers-phonetic-uptodate:


jar-analyzers-phonetic:


check-analyzers-kuromoji-uptodate:


jar-analyzers-kuromoji:


check-codecs-uptodate:


jar-codecs:


check-expressions-uptodate:


jar-expressions:


check-suggest-uptodate:


jar-suggest:


check-highlighter-uptodate:


jar-highlighter:


check-memory-uptodate:


jar-memory:


check-misc-uptodate:


jar-misc:


check-spatial-extras-uptodate:


jar-spatial-extras:


check-grouping-uptodate:


jar-grouping:


check-queries-uptodate:


jar-queries:


check-queryparser-uptodate:


jar-queryparser:


check-join-uptodate:


jar-join:


check-sandbox-uptodate:


jar-sandbox:


check-classification-uptodate:


jar-classification:


prep-lucene-jars:


resolve-example:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


resolve-server:


ivy-availability-check:


ivy-fail:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-fail:


resolve:


common.init:


compile-lucene-core:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


init:


common.compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/classes/java

    [javac] Compiling 5 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/classes/java

    [javac] Note: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/velocity/src/java/org/apache/solr/response/VelocityResponseWriter.java uses or overrides a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

    [javac] Note: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/velocity/src/java/org/apache/solr/response/VelocityResponseWriter.java uses unchecked or unsafe operations.

    [javac] Note: Recompile with -Xlint:unchecked for details.

     [copy] Copying 13 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/classes/java


common-solr.compile-core:


compile-core:


compile-test-framework:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


compile-lucene-core:


compile-codecs:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


common.init:


compile-lucene-core:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


common.compile-core:


compile-core:


common.compile-test:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/classes/test

    [javac] Compiling 2 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/classes/test

     [copy] Copying 2 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/classes/test


compile-test:


install-junit4-taskdef:


compile-tools:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


compile-core:


validate:


resolve-groovy:


-init-totals:


-test:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/test

[loadresource] Do not set property tests.explicitclass as its length is 0.

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/test/temp

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/.caches/test-stats/solr-velocity

   [junit4] <JUnit4> says ¡Hola! Master seed: CFC9244F129B0874

   [junit4] Executing 1 suite with 1 JVM.

   [junit4] 

   [junit4] Started J0 PID(12950@lucene1-us-west).

   [junit4] Suite: org.apache.solr.velocity.VelocityResponseWriterTest

   [junit4] Completed [1/1] in 2.27s, 13 tests

   [junit4] 

   [junit4] JVM J0:     0.29 ..     3.15 =     2.85s

   [junit4] Execution time total: 3.15 sec.

   [junit4] Tests summary: 1 suite, 13 tests

   [junit4] Could not remove temporary path: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/test/J0 (/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-velocity/test/J0)

     [echo] 5 slowest tests:

[junit4:tophints]   2.27s | org.apache.solr.velocity.VelocityResponseWriterTest


-check-totals:


common.test:


-check-totals:


test:


ivy-availability-check:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve-groovy:


-check-totals:


BUILD FAILED

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:

/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:715: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:

* java_pid13866.hprof


Total time: 395 minutes 7 seconds

Build step 'Invoke Ant' marked build as failure

Archiving artifacts

Recording test results

Email was triggered for: Failure - Any

Sending email for trigger: Failure - Any





[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 132 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/132/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Timeout occured while waiting response from server at: https://127.0.0.1:36105

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: https://127.0.0.1:36105
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:601)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:170)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
	at sun.security.ssl.InputRecord.read(InputRecord.java:503)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:930)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:160)
	at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:84)
	at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:273)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261)
	at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:283)
	at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:251)
	at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:197)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124)
	at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:685)
	at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:487)
	at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:882)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:495)
	... 47 more


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 4 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 4 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([AF0C4F5D35A09767]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor43.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
Started by timer

[EnvInject] - Loading node environment variables.

Building remotely on lucene in workspace /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x

 > git rev-parse --is-inside-work-tree # timeout=10

Fetching changes from the remote Git repository

 > git config remote.origin.url https://git-wip-us.apache.org/repos/asf/lucene-solr.git # timeout=10

Cleaning workspace

 > git rev-parse --verify HEAD # timeout=10

Resetting working tree

 > git reset --hard # timeout=10

 > git clean -fdx # timeout=10

Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/lucene-solr.git

 > git --version # timeout=10

 > git -c core.askpass=true fetch --tags --progress https://git-wip-us.apache.org/repos/asf/lucene-solr.git +refs/heads/*:refs/remotes/origin/*

 > git rev-parse refs/remotes/origin/branch_6x^{commit} # timeout=10

 > git rev-parse refs/remotes/origin/origin/branch_6x^{commit} # timeout=10

Checking out Revision e3de51be2edd086c29be8fdcfca7f8a5990a640c (refs/remotes/origin/branch_6x)

 > git config core.sparsecheckout # timeout=10

 > git checkout -f e3de51be2edd086c29be8fdcfca7f8a5990a640c

 > git rev-list e3de51be2edd086c29be8fdcfca7f8a5990a640c # timeout=10

No emails were triggered.

[Lucene-Solr-NightlyTests-6.x] $ /home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/bin/ant -Dtests.multiplier=2 -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt jenkins-nightly

Buildfile: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml


jenkins-nightly:


-print-java-info:

[java-info] java version "1.8.0_74"

[java-info] Java(TM) SE Runtime Environment (1.8.0_74-b02, Oracle Corporation)

[java-info] Java HotSpot(TM) 64-Bit Server VM (25.74-b02, Oracle Corporation)

[java-info] Test args: []


clean:


clean:


clean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: Apache Ivy 2.3.0 - 20130110142753 :: http://ant.apache.org/ivy/ ::

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve-groovy:

[ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working

[ivy:cachepath] 	confs: [default]

[ivy:cachepath] 	found org.codehaus.groovy#groovy-all;2.4.6 in public

[ivy:cachepath] :: resolution report :: resolve 88ms :: artifacts dl 1ms

	---------------------------------------------------------------------

	|                  |            modules            ||   artifacts   |

	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|

	---------------------------------------------------------------------

	|      default     |   1   |   0   |   0   |   0   ||   1   |   0   |

	---------------------------------------------------------------------


-test-with-heapdumps-enabled:

     [echo] Java HotSpot(TM) 64-Bit Server VM: Enabling heap dumps on OutOfMemoryError to dir '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps'.

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve-groovy:


test:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


-clover.load:


resolve-groovy:


-init-totals:


test-core:


-clover.disable:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


-clover.load:


-clover.classpath:


-clover.setup:


clover:


-check-git-state:


-git-cleanroot:


-copy-git-state:

     [copy] Copying 1 file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build


git-autoclean:


resolve:


init:


compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java

    [javac] Compiling 782 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java

    [javac] Note: Some input files use or override a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/document/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/bkd/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/compressing/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene60/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/index/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/fst/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/analysis/tokenattributes/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/search/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene62/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/packed/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/mutable/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/perfield/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/analysis/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/analysis/standard/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/store/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/blocktree/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/search/spans/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene54/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/search/similarities/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/automaton/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene53/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene50/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/geo/package-info.class

     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java


compile-test-framework:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


compile-lucene-core:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


compile-codecs:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


common.init:


compile-lucene-core:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java

    [javac] Compiling 66 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/bloom/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/memory/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/simpletext/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blockterms/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blocktreeords/package-info.class

     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


common.compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java

    [javac] Compiling 185 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java

    [javac] Note: Some input files use or override a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/asserting/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/ramonly/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/mockfile/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cheapbastard/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/mockrandom/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cranky/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/compressing/dummy/package-info.class

     [copy] Copying 4 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java


compile-core:


compile-test:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/test

    [javac] Compiling 445 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/test

    [javac] Note: Some input files use or override a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/test


install-junit4-taskdef:


validate:


resolve-groovy:


-init-totals:


-test:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/test

[loadresource] Do not set property tests.explicitclass as its length is 0.

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/test/temp

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/.caches/test-stats/core

   [junit4] <JUnit4> says שלום! Master seed: 17A3DDEE974B3968

   [junit4] Executing 438 suites with 3 JVMs.

   [junit4] 

   [junit4] Started J0 PID(9183@localhost).

   [junit4] Started J2 PID(9177@localhost).

   [junit4] Started J1 PID(9178@localhost).

   [junit4] Suite: org.apache.lucene.search.TestNot

   [junit4] Completed [1/438] on J2 in 0.89s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestMergedIterator

   [junit4] Completed [2/438] on J2 in 0.61s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSortedNumericSortField

   [junit4] Completed [3/438] on J2 in 0.67s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSortRandom

   [junit4] Completed [4/438] on J1 in 2.72s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSetOnce

   [junit4] Completed [5/438] on J1 in 0.19s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestFilterSpans

   [junit4] Completed [6/438] on J1 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanQueryVisitSubscorers

   [junit4] Completed [7/438] on J1 in 0.51s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestMathUtil

   [junit4] Completed [8/438] on J1 in 0.10s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSumDocFreq

   [junit4] Completed [9/438] on J2 in 2.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanations

   [junit4] Completed [10/438] on J0 in 4.34s, 70 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestNotDocIdSet

   [junit4] Completed [11/438] on J0 in 2.64s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestDocIdSetBuilder

   [junit4] Completed [12/438] on J2 in 2.77s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarity2

   [junit4] Completed [13/438] on J0 in 1.17s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestCachingCollector

   [junit4] Completed [14/438] on J0 in 0.04s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestUnicodeUtil

   [junit4] Completed [15/438] on J2 in 1.77s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestCompiledAutomaton

   [junit4] Completed [16/438] on J2 in 1.05s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFixedBitDocIdSet

   [junit4] Completed [17/438] on J0 in 1.85s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestTimSorter

   [junit4] Completed [18/438] on J0 in 0.27s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestLevenshteinAutomata

   [junit4] Completed [19/438] on J0 in 4.97s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestQueryBuilder

   [junit4] Completed [20/438] on J0 in 0.09s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.Test2BPagedBytes

   [junit4] IGNOR/A 0.01s J0 | Test2BPagedBytes.test

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=You must increase heap to > 2 G to run this))

   [junit4] Completed [21/438] on J0 in 0.02s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocCount

   [junit4] Completed [22/438] on J0 in 0.69s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestAutomaton

   [junit4] Completed [23/438] on J1 in 14.40s, 56 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminizeLexicon

   [junit4] Completed [24/438] on J1 in 2.28s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.packed.TestDirectMonotonic

   [junit4] Completed [25/438] on J0 in 5.51s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestIntroSorter

   [junit4] Completed [26/438] on J0 in 0.09s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTragicIndexWriterDeadlock

   [junit4] Completed [27/438] on J0 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSloppyMath

   [junit4] Completed [28/438] on J1 in 1.53s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCharsRefBuilder

   [junit4] Completed [29/438] on J1 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDoc

   [junit4] Completed [30/438] on J1 in 0.33s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressIndexing

   [junit4] Completed [31/438] on J0 in 4.68s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCustomNorms

   [junit4] Completed [32/438] on J0 in 7.09s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTotalHitCountCollector

   [junit4] Completed [33/438] on J0 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestExitableDirectoryReader

   [junit4] Completed [34/438] on J0 in 0.57s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCloseableThreadLocal

   [junit4] Completed [35/438] on J0 in 0.05s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFilterWeight

   [junit4] Completed [36/438] on J0 in 0.05s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiLevelSkipList

   [junit4] Completed [37/438] on J0 in 0.67s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSmallFloat

   [junit4] Completed [38/438] on J0 in 0.34s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMergePolicy

   [junit4] Completed [39/438] on J0 in 5.55s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.TestCodecUtil

   [junit4] Completed [40/438] on J0 in 0.08s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMergeRateLimiter

   [junit4] Completed [41/438] on J0 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestComplexExplanationsOfNonMatches

   [junit4] Completed [42/438] on J0 in 0.22s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestOfflineSorter

   [junit4] Completed [43/438] on J2 in 31.98s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.packed.TestPackedInts

   [junit4] Completed [44/438] on J1 in 34.16s, 28 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestPagedBytes

   [junit4] Completed [45/438] on J1 in 4.16s, 4 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTwoPhaseCommitTool

   [junit4] Completed [46/438] on J1 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestMultiMMap

   [junit4] Completed [47/438] on J2 in 19.06s, 56 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermVectorsWriter

   [junit4] Completed [48/438] on J1 in 0.53s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCodecs

   [junit4] Completed [49/438] on J2 in 0.73s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.mutable.TestMutableValues

   [junit4] Completed [50/438] on J2 in 0.07s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestIOUtils

   [junit4] Completed [51/438] on J2 in 0.16s, 18 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestReaderWrapperDVTypeCheck

   [junit4] Completed [52/438] on J2 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveChecksumFooter

   [junit4] Completed [53/438] on J1 in 1.64s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOmitTf

   [junit4] Completed [54/438] on J2 in 1.59s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestBasics

   [junit4] Completed [55/438] on J1 in 4.74s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestBagOfPostings

   [junit4] Completed [56/438] on J2 in 8.16s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery

   [junit4] Completed [57/438] on J2 in 3.39s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocInverterPerFieldErrorInfo

   [junit4] Completed [58/438] on J2 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCollectionUtil

   [junit4] Completed [59/438] on J2 in 16.44s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestAutomatonQueryUnicode

   [junit4] Completed [60/438] on J2 in 0.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterDelete

   [junit4] IGNOR/A 0.00s J0 | TestIndexWriterDelete.testIndexingThenDeleting

   [junit4]    > Assumption #1: This test cannot run with Memory codec

   [junit4] Completed [61/438] on J0 in 59.94s, 27 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestRateLimiter

   [junit4] Completed [62/438] on J0 in 1.47s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCrash

   [junit4] Completed [63/438] on J0 in 1.22s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCachingTokenFilter

   [junit4] Completed [64/438] on J0 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnJRECrash

   [junit4] Completed [65/438] on J0 in 4.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestLegacyNumericUtils

   [junit4] Completed [66/438] on J0 in 1.39s, 12 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50CompoundFormat

   [junit4] Completed [67/438] on J0 in 4.74s, 26 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDateSort

   [junit4] Completed [68/438] on J0 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestCharTermAttributeImpl

   [junit4] Completed [69/438] on J0 in 0.38s, 12 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.LimitedFiniteStringsIteratorTest

   [junit4] Completed [70/438] on J0 in 0.15s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestHugeRamFile

   [junit4] Completed [71/438] on J0 in 1.69s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestReaderClosed

   [junit4] Completed [72/438] on J0 in 0.11s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestBytesRef

   [junit4] Completed [73/438] on J0 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPersistentSnapshotDeletionPolicy

   [junit4] Completed [74/438] on J0 in 1.57s, 13 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPointValues

   [junit4] Completed [75/438] on J0 in 4.42s, 32 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpansEnum

   [junit4] Completed [76/438] on J0 in 0.12s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMatchAllDocsQuery

   [junit4] Completed [77/438] on J0 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTopFieldCollector

   [junit4] Completed [78/438] on J0 in 0.80s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSwappedIndexFiles

   [junit4] Completed [79/438] on J0 in 0.18s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnDiskFull

   [junit4] Completed [80/438] on J2 in 35.08s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNRTReaderWithThreads

   [junit4] Completed [81/438] on J0 in 3.48s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSparseFixedBitDocIdSet

   [junit4] Completed [82/438] on J2 in 2.52s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFlex

   [junit4] Completed [83/438] on J2 in 0.42s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.bkd.TestDocIdsWriter

   [junit4] Completed [84/438] on J0 in 5.62s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSizeBoundedForceMerge

   [junit4] Completed [85/438] on J0 in 0.08s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestManyFields

   [junit4] Completed [86/438] on J0 in 1.37s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanCollection

   [junit4] Completed [87/438] on J0 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTermScorer

   [junit4] Completed [88/438] on J0 in 0.11s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastDecompressionMode

   [junit4] Completed [89/438] on J2 in 5.16s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestRAMDirectory

   [junit4] IGNOR/A 0.01s J0 | TestRAMDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] IGNOR/A 0.00s J0 | TestRAMDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] Completed [90/438] on J0 in 3.65s, 48 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestInPlaceMergeSorter

   [junit4] Completed [91/438] on J0 in 0.10s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDirectoryReaderReopen

   [junit4] Completed [92/438] on J0 in 3.11s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTermQuery

   [junit4] Completed [93/438] on J0 in 0.04s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMerging

   [junit4] Completed [94/438] on J2 in 8.53s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTerms

   [junit4] Completed [95/438] on J2 in 2.55s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFilterDirectoryReader

   [junit4] Completed [96/438] on J2 in 0.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestByteArrayDataInput

   [junit4] Completed [97/438] on J2 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNeverDelete

   [junit4] Completed [98/438] on J0 in 5.72s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestRegExp

   [junit4] Completed [99/438] on J0 in 0.28s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAtomicUpdate

   [junit4] Completed [100/438] on J2 in 7.23s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestDemo

   [junit4] Completed [101/438] on J2 in 0.05s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestForTooMuchCloning

   [junit4] Completed [102/438] on J2 in 0.36s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:43:08, stalled for 71.8s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] Suite: org.apache.lucene.util.TestIntroSelector

   [junit4] Completed [103/438] on J2 in 2.04s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestForUtil

   [junit4] Completed [104/438] on J2 in 0.18s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTieredMergePolicy

   [junit4] Completed [105/438] on J2 in 14.17s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestWordlistLoader

   [junit4] Completed [106/438] on J2 in 0.04s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.FiniteStringsIteratorTest

   [junit4] Completed [107/438] on J2 in 1.18s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanNearQuery

   [junit4] Completed [108/438] on J2 in 0.07s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiTermsEnum

   [junit4] Completed [109/438] on J2 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestNativeFSLockFactory

   [junit4] IGNOR/A 0.02s J2 | TestNativeFSLockFactory.testDeleteLockFile

   [junit4]    > Assumption #1: we must be able to delete an open file

   [junit4] IGNOR/A 0.00s J2 | TestNativeFSLockFactory.testStressLocks

   [junit4]    > Assumption #1: cannot handle buggy Files.delete

   [junit4] Completed [110/438] on J2 in 7.20s, 10 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMaxTermFrequency

   [junit4] Completed [111/438] on J2 in 0.36s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMergePolicyWrapper

   [junit4] Completed [112/438] on J2 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanBoostQuery

   [junit4] Completed [113/438] on J2 in 0.05s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanMinShouldMatch

   [junit4] Completed [114/438] on J2 in 4.99s, 17 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanationsOfNonMatches

   [junit4] Completed [115/438] on J2 in 0.40s, 70 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelReaderEmptyIndex

   [junit4] Completed [116/438] on J2 in 0.04s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:44:09, stalled for  132s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:44:09, stalled for 68.4s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterThreadsToSegments

   [junit4] Completed [117/438] on J2 in 61.30s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.fst.TestBytesStore

   [junit4] Completed [118/438] on J2 in 19.91s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOmitPositions

   [junit4] Completed [119/438] on J2 in 0.16s, 4 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:45:09, stalled for  192s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:45:09, stalled for  128s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMaxDocs

   [junit4] IGNOR/A 0.00s J2 | TestIndexWriterMaxDocs.testExactlyAtTrueLimit

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes over two hours))

   [junit4] Completed [120/438] on J2 in 14.80s, 21 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSynonymQuery

   [junit4] Completed [121/438] on J2 in 0.06s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimilarityProvider

   [junit4] Completed [122/438] on J2 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestBufferedIndexInput

   [junit4] Completed [123/438] on J2 in 0.43s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFrequencyTrackingRingBuffer

   [junit4] Completed [124/438] on J2 in 0.63s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCodecHoldsOpenFiles

   [junit4] Completed [125/438] on J2 in 0.12s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestReqExclBulkScorer

   [junit4] Completed [126/438] on J2 in 0.11s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexableField

   [junit4] Completed [127/438] on J2 in 1.83s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.bkd.TestBKD

   [junit4] Completed [128/438] on J2 in 30.03s, 16 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiValuedNumericRangeQuery

   [junit4] Completed [129/438] on J2 in 5.08s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestElevationComparator

   [junit4] Completed [130/438] on J2 in 0.10s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestSimpleFSLockFactory

   [junit4] Completed [131/438] on J2 in 10.80s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BBinaryDocValues

   [junit4] IGNOR/A 0.00s J2 | Test2BBinaryDocValues.testFixedBinary

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))

   [junit4] IGNOR/A 0.00s J2 | Test2BBinaryDocValues.testVariableBinary

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))

   [junit4] Completed [132/438] on J2 in 0.09s, 2 tests, 2 skipped

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:46:09, stalled for  252s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:46:09, stalled for  188s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.index.TestConsistentFieldNumbers

   [junit4] Completed [133/438] on J2 in 11.07s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestPriorityQueue

   [junit4] Completed [134/438] on J2 in 0.22s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestNumericRangeQuery32

   [junit4] Completed [135/438] on J2 in 41.51s, 23 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestLongPostings

   [junit4] Completed [136/438] on J2 in 5.93s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiCollector

   [junit4] Completed [137/438] on J2 in 2.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestWildcard

   [junit4] Completed [138/438] on J2 in 0.19s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimilarity

   [junit4] Completed [139/438] on J2 in 0.22s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPayloads

   [junit4] Completed [140/438] on J2 in 0.33s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRecyclingByteBlockAllocator

   [junit4] Completed [141/438] on J2 in 0.08s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanFirstQuery

   [junit4] Completed [142/438] on J2 in 0.17s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanOr

   [junit4] Completed [143/438] on J2 in 3.41s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSort

   [junit4] Completed [144/438] on J2 in 0.59s, 25 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:47:09, stalled for  249s at: TestSearcherManager.testSearcherManager

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:47:09, stalled for  312s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:48:09, stalled for  309s at: TestSearcherManager.testSearcherManager

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:48:09, stalled for  372s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:48:21, stalled for 71.5s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] Suite: org.apache.lucene.search.TestSearcherManager

   [junit4] Completed [145/438] on J0 in 330.36s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPhraseQuery

   [junit4] Completed [146/438] on J0 in 1.12s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestBytesRefArray

   [junit4] Completed [147/438] on J0 in 0.24s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPositions

   [junit4] IGNOR/A 0.00s J0 | Test2BPositions.test

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=uses lots of space and takes a few minutes))

   [junit4] Completed [148/438] on J0 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocValuesIndexing

   [junit4] Completed [149/438] on J0 in 0.55s, 34 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRecyclingIntBlockAllocator

   [junit4] Completed [150/438] on J0 in 0.19s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanationsWithFillerDocs

   [junit4] IGNOR/A 0.01s J0 | TestSimpleExplanationsWithFillerDocs.testMA2

   [junit4]    > Assumption #1: got: <[test is not viable with empty filler docs, null]>, expected: each not null

   [junit4] IGNOR/A 0.00s J0 | TestSimpleExplanationsWithFillerDocs.testMA1

   [junit4]    > Assumption #1: got: <[test is not viable with empty filler docs, null]>, expected: each not null

   [junit4] Completed [151/438] on J0 in 7.50s, 70 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelCompositeReader

   [junit4] Completed [152/438] on J0 in 0.78s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPrefixQuery

   [junit4] Completed [153/438] on J0 in 10.42s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanMultiTermQueryWrapper

   [junit4] Completed [154/438] on J0 in 0.08s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTermRangeQuery

   [junit4] Completed [155/438] on J0 in 0.22s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermVectorsReader

   [junit4] Completed [156/438] on J0 in 0.34s, 13 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSPIClassIterator

   [junit4] Completed [157/438] on J0 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestDirectory

   [junit4] Completed [158/438] on J0 in 0.53s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMatchNoDocsQuery

   [junit4] Completed [159/438] on J0 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.similarities.TestClassicSimilarity

   [junit4] Completed [160/438] on J0 in 0.11s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexReaderClose

   [junit4] Completed [161/438] on J0 in 1.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriter

   [junit4] Completed [162/438] on J0 in 9.86s, 71 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:49:09, stalled for  432s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:49:21, stalled for  132s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:50:09, stalled for  492s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:50:21, stalled for  192s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat

   [junit4] Completed [163/438] on J0 in 83.53s, 96 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:51:09, stalled for  552s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterReader

   [junit4] Completed [164/438] on J0 in 42.46s, 23 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:51:21, stalled for  252s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:52:09, stalled for  612s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:52:21, stalled for  312s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:52:45, stalled for 60.1s at: TestFSTs.testBigSet

   [junit4] Suite: org.apache.lucene.util.fst.TestFSTs

   [junit4] Completed [165/438] on J0 in 96.63s, 20 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestNumericRangeQuery64

   [junit4] Completed [166/438] on J0 in 11.25s, 29 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads

   [junit4] IGNOR/A 0.01s J0 | TestIndexWriterWithThreads.testOpenTwoIndexWritersOnDifferentThreads

   [junit4]    > Assumption #1: aborting test: timeout obtaining lock

   [junit4] Completed [167/438] on J0 in 9.42s, 12 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight

   [junit4] Completed [168/438] on J1 in 690.93s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestLSBRadixSorter

   [junit4] Completed [169/438] on J1 in 0.08s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAddIndexes

   [junit4] Completed [170/438] on J0 in 6.36s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMinShouldMatch2

   [junit4] Completed [171/438] on J1 in 3.64s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNewestSegment

   [junit4] Completed [172/438] on J1 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestVirtualMethod

   [junit4] Completed [173/438] on J1 in 0.07s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterConfig

   [junit4] Completed [174/438] on J1 in 0.11s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestHighCompressionMode

   [junit4] Completed [175/438] on J1 in 3.69s, 7 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:53:21, stalled for  372s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] Suite: org.apache.lucene.search.TestShardSearching

   [junit4] Completed [176/438] on J0 in 16.55s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDeletionPolicy

   [junit4] Completed [177/438] on J0 in 5.00s, 6 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:54:21, stalled for  432s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:54:33, stalled for 63.9s at: TestMixedDocValuesUpdates.testTonsOfUpdates

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:54:45, stalled for 69.0s at: TestNRTThreads.testNRTThreads

   [junit4] Suite: org.apache.lucene.index.TestMixedDocValuesUpdates

   [junit4] Completed [178/438] on J1 in 115.25s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestConstantScoreQuery

   [junit4] Completed [179/438] on J1 in 0.06s, 5 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:55:21, stalled for  492s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:55:45, stalled for  129s at: TestNRTThreads.testNRTThreads

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T05:56:21, stalled for  552s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:56:33, stalled for 67.4s at: TestIndexSorting.testRandom3

   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterStallControl

   [junit4] Completed [180/438] on J2 in 569.89s, 3 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:56:46, stalled for  189s at: TestNRTThreads.testNRTThreads

   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecs

   [junit4] Completed [181/438] on J2 in 23.70s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBoolean2

   [junit4] Completed [182/438] on J2 in 3.28s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressAdvance

   [junit4] Completed [183/438] on J2 in 13.12s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNorms

   [junit4] Completed [184/438] on J2 in 5.43s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:57:34, stalled for  127s at: TestIndexSorting.testRandom3

   [junit4] Suite: org.apache.lucene.analysis.standard.TestStandardAnalyzer

   [junit4] Completed [185/438] on J2 in 13.89s, 34 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterCommit

   [junit4] Completed [186/438] on J2 in 2.13s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTimeLimitingCollector

   [junit4] Completed [187/438] on J2 in 1.91s, 7 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:57:46, stalled for  249s at: TestNRTThreads.testNRTThreads

   [junit4] Suite: org.apache.lucene.search.TestSearchWithThreads

   [junit4] Completed [188/438] on J2 in 6.42s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSearchAfter

   [junit4] Completed [189/438] on J2 in 1.45s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestConcurrentMergeScheduler

   [junit4] Completed [190/438] on J2 in 8.59s, 16 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestMinimize

   [junit4] Completed [191/438] on J2 in 1.99s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexSorting

   [junit4] Completed [192/438] on J1 in 189.82s, 27 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestByteBlockPool

   [junit4] Completed [193/438] on J1 in 0.14s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BSortedDocValuesOrds

   [junit4] IGNOR/A 0.00s J1 | Test2BSortedDocValuesOrds.test2BOrds

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=Takes ~ 6 hours if the heap is 5gb))

   [junit4] Completed [194/438] on J1 in 0.01s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestAssertions

   [junit4] Completed [195/438] on J1 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestRollingUpdates

   [junit4] Completed [196/438] on J2 in 22.79s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSnapshotDeletionPolicy

   [junit4] Completed [197/438] on J2 in 1.42s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterUnicode

   [junit4] Completed [198/438] on J2 in 4.69s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterForceMerge

   [junit4] Completed [199/438] on J2 in 4.62s, 4 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T05:58:46, stalled for  309s at: TestNRTThreads.testNRTThreads

   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat

   [junit4] IGNOR/A 0.00s J2 | TestPerFieldPostingsFormat.testPostingsEnumReuse

   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it

   [junit4] IGNOR/A 0.01s J2 | TestPerFieldPostingsFormat.testMergeStability

   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it

   [junit4] Completed [200/438] on J2 in 22.99s, 24 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestArrayUtil

   [junit4] Completed [201/438] on J2 in 4.27s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNRTThreads

   [junit4] Completed [202/438] on J0 in 334.79s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestWeakIdentityMap

   [junit4] Completed [203/438] on J0 in 15.78s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDirectoryReader

   [junit4] Completed [204/438] on J0 in 0.89s, 24 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentMerger

   [junit4] Completed [205/438] on J0 in 0.47s, 3 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T05:59:58, stalled for 63.2s at: TestIndexWriterExceptions.testTooManyTokens

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:00:10, stalled for 65.9s at: TestTermsEnum.testVaryingTermsPerSegment

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:00:34, stalled for 65.0s at: TestBagOfPositions.test

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T06:00:58, stalled for  123s at: TestIndexWriterExceptions.testTooManyTokens

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:01:10, stalled for  126s at: TestTermsEnum.testVaryingTermsPerSegment

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:01:34, stalled for  125s at: TestBagOfPositions.test

   [junit4] Suite: org.apache.lucene.index.TestTermsEnum

   [junit4] Completed [206/438] on J2 in 150.34s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOmitNorms

   [junit4] Completed [207/438] on J2 in 0.63s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestComplexExplanations

   [junit4] Completed [208/438] on J2 in 0.72s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestNRTCachingDirectory

   [junit4] IGNOR/A 0.00s J2 | TestNRTCachingDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] IGNOR/A 0.01s J2 | TestNRTCachingDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] Completed [209/438] on J2 in 2.75s, 46 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterNRTIsCurrent

   [junit4] Completed [210/438] on J2 in 6.10s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFieldCacheRewriteMethod

   [junit4] Completed [211/438] on J2 in 4.15s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiThreadTermVectors

   [junit4] Completed [212/438] on J2 in 1.10s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentTermDocs

   [junit4] Completed [213/438] on J2 in 0.42s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterExceptions

   [junit4] Completed [214/438] on J1 in 208.21s, 36 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRadixSelector

   [junit4] Completed [215/438] on J1 in 0.38s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat2

   [junit4] Completed [216/438] on J2 in 2.71s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50FieldInfoFormat

   [junit4] Completed [217/438] on J1 in 0.78s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDocValuesScoring

   [junit4] Completed [218/438] on J1 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastCompressionMode

   [junit4] Completed [219/438] on J1 in 1.99s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminism

   [junit4] Completed [220/438] on J1 in 0.79s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestSimpleFSDirectory

   [junit4] Completed [221/438] on J1 in 4.20s, 44 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentReader

   [junit4] Completed [222/438] on J1 in 1.52s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiDocValues

   [junit4] Completed [223/438] on J2 in 11.10s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestToken

   [junit4] Completed [224/438] on J2 in 0.01s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFieldsReader

   [junit4] Completed [225/438] on J2 in 0.22s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestCustomSearcherSort

   [junit4] Completed [226/438] on J2 in 1.73s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermsEnum2

   [junit4] Completed [227/438] on J2 in 3.05s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMixedCodecs

   [junit4] Completed [228/438] on J2 in 3.48s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTransactionRollback

   [junit4] Completed [229/438] on J2 in 0.39s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestUTF32ToUTF8

   [junit4] Completed [230/438] on J2 in 7.37s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFixedBitSet

   [junit4] Completed [231/438] on J1 in 19.71s, 19 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestReadOnlyIndex

   [junit4] Completed [232/438] on J1 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSameTokenSamePosition

   [junit4] Completed [233/438] on J1 in 0.05s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestLockFactory

   [junit4] Completed [234/438] on J1 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiset

   [junit4] Completed [235/438] on J1 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressIndexing2

   [junit4] Completed [236/438] on J2 in 1.48s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarityBase

   [junit4] Completed [237/438] on J1 in 1.32s, 32 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiPhraseQuery

   [junit4] Completed [238/438] on J2 in 0.27s, 17 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestThreadedForceMerge

   [junit4] Completed [239/438] on J1 in 0.83s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestLazyProxSkipping

   [junit4] Completed [240/438] on J2 in 0.89s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocsAndPositions

   [junit4] Completed [241/438] on J2 in 1.20s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestMergeSchedulerExternal

   [junit4] Completed [242/438] on J2 in 0.64s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestBytesRefHash

   [junit4] Completed [243/438] on J2 in 3.04s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressNRT

   [junit4] Completed [244/438] on J2 in 0.22s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexInput

   [junit4] Completed [245/438] on J2 in 1.01s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:02:34, stalled for  185s at: TestBagOfPositions.test

   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery2

   [junit4] Completed [246/438] on J2 in 13.76s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDisjunctionMaxQuery

   [junit4] Completed [247/438] on J2 in 0.50s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestForceMergeForever

   [junit4] Completed [248/438] on J2 in 1.37s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestNearSpansOrdered

   [junit4] Completed [249/438] on J2 in 0.46s, 19 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSameScoresWithThreads

   [junit4] Completed [250/438] on J2 in 0.77s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiFields

   [junit4] Completed [251/438] on J2 in 1.86s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocumentWriter

   [junit4] Completed [252/438] on J2 in 0.08s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCrashCausesCorruptIndex

   [junit4] Completed [253/438] on J2 in 0.05s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiTermConstantScore

   [junit4] Completed [254/438] on J2 in 3.68s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestLiveFieldValues

   [junit4] Completed [255/438] on J2 in 0.73s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanSearchEquivalence

   [junit4] Completed [256/438] on J2 in 31.66s, 33 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFuzzyQuery

   [junit4] Completed [257/438] on J2 in 0.21s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestExternalCodecs

   [junit4] Completed [258/438] on J2 in 4.62s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNoDeletionPolicy

   [junit4] Completed [259/438] on J2 in 0.08s, 4 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:03:34, stalled for  245s at: TestBagOfPositions.test

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T06:03:34, stalled for 69.5s at: TestNumericDocValuesUpdates.testTonsOfUpdates

   [junit4] Suite: org.apache.lucene.search.TestRegexpQuery

   [junit4] Completed [260/438] on J2 in 0.16s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPerSegmentDeletes

   [junit4] Completed [261/438] on J2 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIntBlockPool

   [junit4] Completed [262/438] on J2 in 0.44s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPrefixInBooleanQuery

   [junit4] Completed [263/438] on J2 in 2.02s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentTermEnum

   [junit4] Completed [264/438] on J2 in 0.08s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestUniqueTermCount

   [junit4] Completed [265/438] on J2 in 0.08s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanQuery

   [junit4] Completed [266/438] on J2 in 0.42s, 22 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPrefixCodedTerms

   [junit4] Completed [267/438] on J2 in 0.35s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveCodecHeader

   [junit4] Completed [268/438] on J2 in 0.24s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestAutomatonQuery

   [junit4] Completed [269/438] on J2 in 1.30s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestDocument

   [junit4] Completed [270/438] on J2 in 0.09s, 12 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestFileSwitchDirectory

   [junit4] IGNOR/A 0.00s J2 | TestFileSwitchDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] IGNOR/A 0.00s J2 | TestFileSwitchDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] Completed [271/438] on J2 in 1.62s, 45 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSentinelIntSet

   [junit4] Completed [272/438] on J2 in 0.13s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestWildcardRandom

   [junit4] Completed [273/438] on J2 in 0.27s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPrefixRandom

   [junit4] Completed [274/438] on J2 in 1.26s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom

   [junit4] Completed [275/438] on J2 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCheckIndex

   [junit4] Completed [276/438] on J2 in 0.48s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPayloadsOnVectors

   [junit4] Completed [277/438] on J2 in 0.03s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestSearchForDuplicates

   [junit4] Completed [278/438] on J2 in 0.15s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexFileDeleter

   [junit4] Completed [279/438] on J2 in 1.03s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelTermEnum

   [junit4] Completed [280/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestBinaryTerms

   [junit4] Completed [281/438] on J2 in 0.05s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSubScorerFreqs

   [junit4] Completed [282/438] on J2 in 0.04s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestConjunctions

   [junit4] Completed [283/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiTermQueryRewrites

   [junit4] Completed [284/438] on J2 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestField

   [junit4] Completed [285/438] on J2 in 0.05s, 27 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanationsOfNonMatches

   [junit4] Completed [286/438] on J2 in 0.03s, 30 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRollingBuffer

   [junit4] Completed [287/438] on J2 in 0.15s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestNumericTokenStream

   [junit4] Completed [288/438] on J2 in 0.02s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFilterLeafReader

   [junit4] Completed [289/438] on J2 in 0.04s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestSearch

   [junit4] Completed [290/438] on J2 in 0.04s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFilterIterator

   [junit4] Completed [291/438] on J2 in 0.01s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestDateTools

   [junit4] Completed [292/438] on J2 in 0.02s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCharsRef

   [junit4] Completed [293/438] on J2 in 0.03s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestBagOfPositions

   [junit4] Completed [294/438] on J0 in 265.09s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanScorer

   [junit4] Completed [295/438] on J0 in 7.95s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRamUsageEstimator

   [junit4] IGNOR/A 0.00s J0 | TestRamUsageEstimator.testPrintValues

   [junit4]    > Assumption #1: Specify -Dtests.verbose=true to print constants of RamUsageEstimator.

   [junit4] Completed [296/438] on J0 in 0.01s, 5 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIsCurrent

   [junit4] Completed [297/438] on J0 in 0.07s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNoMergePolicy

   [junit4] Completed [298/438] on J0 in 0.02s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNoMergeScheduler

   [junit4] Completed [299/438] on J0 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestNamedSPILoader

   [junit4] Completed [300/438] on J0 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterLockRelease

   [junit4] Completed [301/438] on J0 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPhrasePrefixQuery

   [junit4] Completed [302/438] on J0 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharFilter

   [junit4] Completed [303/438] on J0 in 0.01s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestRollback

   [junit4] Completed [304/438] on J0 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestVersion

   [junit4] Completed [305/438] on J0 in 0.03s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNumericDocValuesUpdates

   [junit4] Completed [306/438] on J1 in 121.51s, 31 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.packed.TestDirectPacked

   [junit4] Completed [307/438] on J1 in 9.44s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermVectors

   [junit4] Completed [308/438] on J1 in 0.43s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMaxPosition

   [junit4] Completed [309/438] on J1 in 0.04s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanations

   [junit4] Completed [310/438] on J1 in 0.45s, 30 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestUsageTrackingFilterCachingPolicy

   [junit4] Completed [311/438] on J1 in 0.03s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestAttributeSource

   [junit4] Completed [312/438] on J1 in 0.07s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestNumericUtils

   [junit4] Completed [313/438] on J1 in 1.41s, 24 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestByteSlices

   [junit4] Completed [314/438] on J1 in 5.14s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTopDocsCollector

   [junit4] Completed [315/438] on J1 in 0.04s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelLeafReader

   [junit4] Completed [316/438] on J1 in 0.32s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.bkd.TestMutablePointsReaderUtils

   [junit4] Completed [317/438] on J1 in 0.94s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleSearchEquivalence

   [junit4] Completed [318/438] on J1 in 9.08s, 17 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom2

   [junit4] Completed [319/438] on J1 in 4.36s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFixedLengthBytesRefArray

   [junit4] Completed [320/438] on J1 in 0.15s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestBinaryDocument

   [junit4] Completed [321/438] on J1 in 0.02s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:04:58, stalled for 66.7s at: Test2BPostings.test

   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterDeleteQueue

   [junit4] Completed [322/438] on J1 in 0.36s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestStringHelper

   [junit4] Completed [323/438] on J1 in 0.04s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSortRescorer

   [junit4] Completed [324/438] on J1 in 0.73s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPostingsOffsets

   [junit4] Completed [325/438] on J1 in 1.99s, 13 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestScorerPerf

   [junit4] Completed [326/438] on J1 in 8.86s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:05:10, stalled for 65.4s at: Test4GBStoredFields.test

   [junit4] Suite: org.apache.lucene.search.spans.TestSpans

   [junit4] Completed [327/438] on J1 in 0.86s, 26 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTopDocsMerge

   [junit4] Completed [328/438] on J1 in 36.75s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTransactions

   [junit4] Completed [329/438] on J1 in 3.27s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:05:58, stalled for  127s at: Test2BPostings.test

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:06:10, stalled for  125s at: Test4GBStoredFields.test

   [junit4] Suite: org.apache.lucene.util.TestTimSorterWorstCase

   [junit4] Completed [330/438] on J1 in 26.31s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestNIOFSDirectory

   [junit4] Completed [331/438] on J1 in 3.97s, 44 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPositionIncrement

   [junit4] Completed [332/438] on J1 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestFieldMaskingSpanQuery

   [junit4] IGNOR/A 0.00s J1 | TestFieldMaskingSpanQuery.testSimple2

   [junit4]    > Assumption #1: Broken scoring: LUCENE-3723

   [junit4] IGNOR/A 0.00s J1 | TestFieldMaskingSpanQuery.testSpans2

   [junit4]    > Assumption #1: Broken scoring: LUCENE-3723

   [junit4] Completed [333/438] on J1 in 0.22s, 11 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFlushByRamOrCountsPolicy

   [junit4] Completed [334/438] on J1 in 7.18s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.MultiCollectorTest

   [junit4] Completed [335/438] on J1 in 0.01s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDocBoost

   [junit4] Completed [336/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestNGramPhraseQuery

   [junit4] Completed [337/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestScoreCachingWrappingScorer

   [junit4] Completed [338/438] on J1 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestIntsRef

   [junit4] Completed [339/438] on J1 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestSimpleAttributeImpl

   [junit4] Completed [340/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermdocPerf

   [junit4] Completed [341/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexCommit

   [junit4] Completed [342/438] on J1 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTerm

   [junit4] Completed [343/438] on J1 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPostingsBytes

   [junit4] IGNOR/A 0.00s J1 | Test2BPostingsBytes.test

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~20GB-30GB of space and 10 minutes))

   [junit4] Completed [344/438] on J1 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPositiveScoresOnlyCollector

   [junit4] Completed [345/438] on J1 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BTerms

   [junit4] IGNOR/A 0.00s J1 | Test2BTerms.test2BTerms

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=very slow, use 5g minimum heap))

   [junit4] Completed [346/438] on J1 in 0.01s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BNumericDocValues

   [junit4] IGNOR/A 0.00s J1 | Test2BNumericDocValues.testNumerics

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 2 hours if the heap is 5gb))

   [junit4] Completed [347/438] on J1 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharArrayMap

   [junit4] Completed [349/438] on J1 in 0.08s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharArraySet

   [junit4] Completed [350/438] on J1 in 0.02s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharacterUtils

   [junit4] Completed [351/438] on J1 in 0.02s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestReusableStringReader

   [junit4] Completed [352/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestStopFilter

   [junit4] Completed [353/438] on J1 in 0.01s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestBytesRefAttImpl

   [junit4] Completed [354/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestPackedTokenAttributeImpl

   [junit4] Completed [355/438] on J1 in 0.01s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.TestCodecLoadingDeadlock

   [junit4] Completed [356/438] on J1 in 0.32s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:06:58, stalled for  187s at: Test2BPostings.test

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:07:10, stalled for  185s at: Test4GBStoredFields.test

   [junit4] Suite: org.apache.lucene.index.Test4GBStoredFields

   [junit4] Completed [357/438] on J0 in 211.53s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestGrowableByteArrayDataOutput

   [junit4] Completed [358/438] on J1 in 67.76s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2

   [junit4] Completed [359/438] on J1 in 3.19s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat3

   [junit4] Completed [360/438] on J1 in 8.23s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:07:58, stalled for  247s at: Test2BPostings.test

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat

   [junit4] Completed [361/438] on J0 in 23.19s, 26 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormat

   [junit4] Completed [362/438] on J1 in 14.11s, 17 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormatHighCompression

   [junit4] Completed [363/438] on J0 in 18.70s, 19 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50TermVectorsFormat

   [junit4] Completed [364/438] on J1 in 21.76s, 17 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:08:58, stalled for  307s at: Test2BPostings.test

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:09:58, stalled for 67.4s at: TestLucene53NormsFormat.testNCommonBig

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:09:58, stalled for  367s at: Test2BPostings.test

   [junit4] Suite: org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat

   [junit4] IGNOR/A 0.00s J0 | TestLucene53NormsFormat.testMergeStability

   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it

   [junit4] Completed [365/438] on J0 in 108.33s, 17 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene60.TestLucene60PointsFormat

   [junit4] IGNOR/A 0.00s J0 | TestLucene60PointsFormat.testMergeStability

   [junit4]    > Assumption #1: merge is not stable

   [junit4] Completed [366/438] on J0 in 13.49s, 17 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene62.TestLucene62SegmentInfoFormat

   [junit4] Completed [367/438] on J0 in 0.26s, 16 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestFieldType

   [junit4] Completed [368/438] on J0 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestGeoEncodingUtils

   [junit4] Completed [369/438] on J0 in 0.50s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestGeoUtils

   [junit4] Completed [370/438] on J0 in 18.53s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestPolygon

   [junit4] Completed [371/438] on J0 in 0.03s, 15 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(9177@localhost): 2016-07-31T06:10:58, stalled for  427s at: Test2BPostings.test

   [junit4] Suite: org.apache.lucene.geo.TestPolygon2D

   [junit4] Completed [372/438] on J0 in 22.72s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPoints

   [junit4] IGNOR/A 0.01s J0 | Test2BPoints.test1D

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))

   [junit4] IGNOR/A 0.00s J0 | Test2BPoints.test2D

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))

   [junit4] Completed [373/438] on J0 in 0.01s, 2 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BSortedDocValuesFixedSorted

   [junit4] IGNOR/A 0.00s J0 | Test2BSortedDocValuesFixedSorted.testFixedSorted

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=Takes ~ 6 hours if the heap is 5gb))

   [junit4] Completed [374/438] on J0 in 0.01s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPostings

   [junit4] Completed [375/438] on J2 in 435.89s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesCheckIndexHeader

   [junit4] Completed [376/438] on J0 in 4.59s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesDetectTruncation

   [junit4] Completed [377/438] on J2 in 8.70s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(9178@localhost): 2016-07-31T06:12:10, stalled for 67.4s at: TestLucene54DocValuesFormat.testSortedSetVariableLengthBigVsStoredFields

   [junit4] HEARTBEAT J0 PID(9183@localhost): 2016-07-31T06:12:10, stalled for 61.1s at: TestBinaryDocValuesUpdates.testTonsOfUpdates

   [junit4] Suite: org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat

   [junit4] Completed [378/438] on J1 in 246.98s, 109 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocIDMerger

   [junit4] Completed [379/438] on J1 in 0.04s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocValues

   [junit4] Completed [380/438] on J1 in 0.09s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestExceedMaxTermLength

   [junit4] Completed [381/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFieldReuse

   [junit4] Completed [3

[...truncated too long message...]

-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160731_081509_572.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J1-20160731_081509_572.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 271 minutes 13 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {solr/build/solr-core/test/temp/junit4-J2-20160731_081509_573.events=solr/build/solr-core/test/temp/junit4-J2-20160731_081509_573.events, README.txt=README.txt, heapdumps/java_pid14710.hprof=heapdumps/java_pid14710.hprof, solr/build/solr-core/test/temp/junit4-J0-20160731_081509_571.events=solr/build/solr-core/test/temp/junit4-J0-20160731_081509_571.events, solr/build/solr-core/test/temp/junit4-J1-20160731_081509_572.events=solr/build/solr-core/test/temp/junit4-J1-20160731_081509_572.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any





[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 131 - Still Failing

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/131/

2 tests failed.
FAILED:  org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test

Error Message:
Error from server at http://127.0.0.1:58800: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: KeeperErrorCode = Session expired for /overseer/queue/qn-

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58800: Error CREATEing SolrCore 'halfcollection_shard1_replica1': Unable to create core [halfcollection_shard1_replica1] Caused by: KeeperErrorCode = Session expired for /overseer/queue/qn-
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:590)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:259)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.makeRequest(CollectionsAPIDistributedZkTest.java:399)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testErrorHandling(CollectionsAPIDistributedZkTest.java:515)
	at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.test(CollectionsAPIDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler

Error Message:
ObjectTracker found 2 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory]

Stack Trace:
java.lang.AssertionError: ObjectTracker found 2 object(s) that were not released!!! [NRTCachingDirectory, NRTCachingDirectory]
	at __randomizedtesting.SeedInfo.seed([F80A11D6CEA15901]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNull(Assert.java:551)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:258)
	at sun.reflect.GeneratedMethodAccessor40.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:834)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
Started by timer

[EnvInject] - Loading node environment variables.

Building remotely on lucene in workspace /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x

 > git rev-parse --is-inside-work-tree # timeout=10

Fetching changes from the remote Git repository

 > git config remote.origin.url https://git-wip-us.apache.org/repos/asf/lucene-solr.git # timeout=10

Cleaning workspace

 > git rev-parse --verify HEAD # timeout=10

Resetting working tree

 > git reset --hard # timeout=10

 > git clean -fdx # timeout=10

Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/lucene-solr.git

 > git --version # timeout=10

 > git -c core.askpass=true fetch --tags --progress https://git-wip-us.apache.org/repos/asf/lucene-solr.git +refs/heads/*:refs/remotes/origin/*

 > git rev-parse refs/remotes/origin/branch_6x^{commit} # timeout=10

 > git rev-parse refs/remotes/origin/origin/branch_6x^{commit} # timeout=10

Checking out Revision e3de51be2edd086c29be8fdcfca7f8a5990a640c (refs/remotes/origin/branch_6x)

 > git config core.sparsecheckout # timeout=10

 > git checkout -f e3de51be2edd086c29be8fdcfca7f8a5990a640c

 > git rev-list 6b8b34ed358287a1cadf848e14089601a3ce1671 # timeout=10

No emails were triggered.

[Lucene-Solr-NightlyTests-6.x] $ /home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/bin/ant -Dtests.multiplier=2 -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt jenkins-nightly

Buildfile: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml


jenkins-nightly:


-print-java-info:

[java-info] java version "1.8.0_74"

[java-info] Java(TM) SE Runtime Environment (1.8.0_74-b02, Oracle Corporation)

[java-info] Java HotSpot(TM) 64-Bit Server VM (25.74-b02, Oracle Corporation)

[java-info] Test args: []


clean:


clean:


clean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: Apache Ivy 2.3.0 - 20130110142753 :: http://ant.apache.org/ivy/ ::

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve-groovy:

[ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working

[ivy:cachepath] 	confs: [default]

[ivy:cachepath] 	found org.codehaus.groovy#groovy-all;2.4.6 in public

[ivy:cachepath] :: resolution report :: resolve 90ms :: artifacts dl 1ms

	---------------------------------------------------------------------

	|                  |            modules            ||   artifacts   |

	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|

	---------------------------------------------------------------------

	|      default     |   1   |   0   |   0   |   0   ||   1   |   0   |

	---------------------------------------------------------------------


-test-with-heapdumps-enabled:

     [echo] Java HotSpot(TM) 64-Bit Server VM: Enabling heap dumps on OutOfMemoryError to dir '/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps'.

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/heapdumps


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve-groovy:


test:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


-clover.load:


resolve-groovy:


-init-totals:


test-core:


-clover.disable:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


-clover.load:


-clover.classpath:


-clover.setup:


clover:


-check-git-state:


-git-cleanroot:


-copy-git-state:

     [copy] Copying 1 file to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build


git-autoclean:


resolve:


init:


compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java

    [javac] Compiling 782 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java

    [javac] Note: Some input files use or override a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/document/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/bkd/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/compressing/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene60/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/index/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/fst/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/analysis/tokenattributes/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/search/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene62/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/packed/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/mutable/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/perfield/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/analysis/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/analysis/standard/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/store/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/blocktree/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/search/spans/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene54/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/search/similarities/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/util/automaton/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene53/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/codecs/lucene50/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java/org/apache/lucene/geo/package-info.class

     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/java


compile-test-framework:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


compile-lucene-core:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:


compile-codecs:


-check-git-state:


-git-cleanroot:


-copy-git-state:


git-autoclean:


ivy-availability-check:


ivy-fail:


ivy-configure:

[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/top-level-ivy-settings.xml


resolve:


common.init:


compile-lucene-core:


init:


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java

    [javac] Compiling 66 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/bloom/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/memory/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/simpletext/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blockterms/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java/org/apache/lucene/codecs/blocktreeords/package-info.class

     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/codecs/classes/java


-clover.disable:


-clover.load:


-clover.classpath:


-clover.setup:


clover:


common.compile-core:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java

    [javac] Compiling 185 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java

    [javac] Note: Some input files use or override a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/asserting/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/ramonly/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/mockfile/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cheapbastard/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/mockrandom/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/cranky/package-info.class

    [javac] Creating empty /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java/org/apache/lucene/codecs/compressing/dummy/package-info.class

     [copy] Copying 4 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/test-framework/classes/java


compile-core:


compile-test:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/test

    [javac] Compiling 445 source files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/test

    [javac] Note: Some input files use or override a deprecated API.

    [javac] Note: Recompile with -Xlint:deprecation for details.

     [copy] Copying 3 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/classes/test


install-junit4-taskdef:


validate:


resolve-groovy:


-init-totals:


-test:

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/test

[loadresource] Do not set property tests.explicitclass as its length is 0.

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/core/test/temp

    [mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/.caches/test-stats/core

   [junit4] <JUnit4> says 今日は! Master seed: 24F0309E0F1E5BCB

   [junit4] Executing 438 suites with 3 JVMs.

   [junit4] 

   [junit4] Started J0 PID(30154@localhost).

   [junit4] Started J2 PID(30170@localhost).

   [junit4] Started J1 PID(30169@localhost).

   [junit4] Suite: org.apache.lucene.index.Test2BSortedDocValuesOrds

   [junit4] IGNOR/A 0.01s J1 | Test2BSortedDocValuesOrds.test2BOrds

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=Takes ~ 6 hours if the heap is 5gb))

   [junit4] Completed [1/438] on J1 in 0.19s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestIndexSearcher

   [junit4] Completed [2/438] on J1 in 1.59s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestStringHelper

   [junit4] Completed [3/438] on J1 in 0.04s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCrash

   [junit4] Completed [4/438] on J0 in 2.74s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAtomicUpdate

   [junit4] Completed [5/438] on J2 in 7.88s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestUpgradeIndexMergePolicy

   [junit4] Completed [6/438] on J2 in 0.23s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestPriorityQueue

   [junit4] Completed [7/438] on J2 in 1.05s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterDeleteByQuery

   [junit4] Completed [8/438] on J2 in 0.36s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterUnicode

   [junit4] Completed [10/438] on J1 in 8.93s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFieldCacheRewriteMethod

   [junit4] Completed [11/438] on J0 in 15.54s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRecyclingIntBlockAllocator

   [junit4] Completed [12/438] on J0 in 0.09s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIntBlockPool

   [junit4] Completed [13/438] on J0 in 1.48s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestUTF32ToUTF8

   [junit4] Completed [14/438] on J0 in 10.18s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNRTReaderCleanup

   [junit4] Completed [15/438] on J0 in 0.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNRTReaderWithThreads

   [junit4] Completed [16/438] on J0 in 1.78s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestCachingCollector

   [junit4] Completed [17/438] on J0 in 0.24s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestBytesRefArray

   [junit4] Completed [18/438] on J0 in 0.58s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestSingleInstanceLockFactory

   [junit4] Completed [19/438] on J0 in 3.34s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocValues

   [junit4] Completed [20/438] on J0 in 0.42s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterReader

   [junit4] Completed [21/438] on J2 in 58.81s, 23 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.fst.TestFSTs

   [junit4] Completed [22/438] on J1 in 64.84s, 20 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNoMergePolicy

   [junit4] Completed [23/438] on J1 in 0.08s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanSearchEquivalence

   [junit4] Completed [24/438] on J0 in 47.71s, 33 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSentinelIntSet

   [junit4] Completed [25/438] on J0 in 0.17s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFieldsReader

   [junit4] Completed [26/438] on J0 in 0.72s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTotalHitCountCollector

   [junit4] Completed [27/438] on J0 in 0.15s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestComplexExplanations

   [junit4] Completed [28/438] on J0 in 1.82s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestMinimize

   [junit4] Completed [29/438] on J0 in 2.47s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.FuzzyTermOnShortTermsTest

   [junit4] Completed [30/438] on J0 in 0.27s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestConjunctionDISI

   [junit4] Completed [31/438] on J0 in 10.07s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentTermDocs

   [junit4] Completed [32/438] on J0 in 0.72s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestSimpleAttributeImpl

   [junit4] Completed [33/438] on J0 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestNGramPhraseQuery

   [junit4] Completed [34/438] on J0 in 0.23s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestQueryBuilder

   [junit4] Completed [35/438] on J0 in 0.08s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestHugeRamFile

   [junit4] Completed [36/438] on J0 in 1.78s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPersistentSnapshotDeletionPolicy

   [junit4] Completed [37/438] on J0 in 1.36s, 13 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNumericDocValuesUpdates

   [junit4] Completed [38/438] on J1 in 29.08s, 31 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanQueryVisitSubscorers

   [junit4] Completed [39/438] on J1 in 0.23s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFixedLengthBytesRefArray

   [junit4] Completed [40/438] on J1 in 0.68s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCharsRef

   [junit4] Completed [41/438] on J1 in 0.07s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocIDMerger

   [junit4] Completed [42/438] on J1 in 0.07s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestFileSwitchDirectory

   [junit4] IGNOR/A 0.01s J1 | TestFileSwitchDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] IGNOR/A 0.00s J1 | TestFileSwitchDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] Completed [43/438] on J1 in 2.96s, 45 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestVersion

   [junit4] Completed [44/438] on J1 in 0.07s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFieldReuse

   [junit4] Completed [45/438] on J1 in 0.03s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterWithThreads

   [junit4] IGNOR/A 0.04s J0 | TestIndexWriterWithThreads.testOpenTwoIndexWritersOnDifferentThreads

   [junit4]    > Assumption #1: aborting test: timeout obtaining lock

   [junit4] Completed [46/438] on J0 in 9.44s, 12 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesDetectTruncation

   [junit4] Completed [47/438] on J0 in 2.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestBasics

   [junit4] Completed [48/438] on J1 in 7.00s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanRewrites

   [junit4] Completed [49/438] on J1 in 0.95s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.bkd.Test2BBKDPoints

   [junit4] IGNOR/A 0.00s J1 | Test2BBKDPoints.test2D

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))

   [junit4] IGNOR/A 0.00s J1 | Test2BBKDPoints.test1D

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))

   [junit4] Completed [50/438] on J1 in 0.03s, 2 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.MultiCollectorTest

   [junit4] Completed [51/438] on J1 in 0.06s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPointValues

   [junit4] Completed [52/438] on J0 in 12.37s, 32 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestTimSorter

   [junit4] Completed [53/438] on J0 in 0.32s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSwappedIndexFiles

   [junit4] Completed [54/438] on J0 in 0.20s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDateSort

   [junit4] Completed [55/438] on J0 in 0.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestNotDocIdSet

   [junit4] Completed [56/438] on J0 in 2.38s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestIntroSorter

   [junit4] Completed [57/438] on J0 in 0.18s, 9 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:01:37, stalled for 63.2s at: TestIndexWriterThreadsToSegments.testSegmentCountOnFlushRandom

   [junit4] Suite: org.apache.lucene.index.TestIndexWriter

   [junit4] Completed [58/438] on J0 in 13.06s, 71 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMatchAllDocsQuery

   [junit4] Completed [59/438] on J0 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTragicIndexWriterDeadlock

   [junit4] Completed [60/438] on J0 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocCount

   [junit4] Completed [61/438] on J0 in 0.38s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarity2

   [junit4] Completed [62/438] on J0 in 0.57s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterThreadsToSegments

   [junit4] Completed [63/438] on J2 in 78.09s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSumDocFreq

   [junit4] Completed [64/438] on J2 in 4.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestSimpleFSLockFactory

   [junit4] Completed [65/438] on J2 in 9.40s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestFilterDirectory

   [junit4] IGNOR/A 0.01s J2 | TestFilterDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] IGNOR/A 0.01s J2 | TestFilterDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] Completed [66/438] on J2 in 1.48s, 46 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestConcurrentMergeScheduler

   [junit4] Completed [67/438] on J2 in 17.45s, 16 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.similarities.TestBM25Similarity

   [junit4] Completed [68/438] on J2 in 0.08s, 3 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:02:25, stalled for 62.6s at: TestControlledRealTimeReopenThread.testControlledRealTimeReopenThread

   [junit4] Suite: org.apache.lucene.util.TestRoaringDocIdSet

   [junit4] Completed [69/438] on J2 in 1.07s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocumentWriter

   [junit4] Completed [70/438] on J2 in 0.19s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery

   [junit4] Completed [71/438] on J2 in 3.14s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTerm

   [junit4] Completed [72/438] on J2 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestOperations

   [junit4] Completed [73/438] on J2 in 1.84s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestRollingUpdates

   [junit4] Completed [74/438] on J2 in 28.98s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermsEnum2

   [junit4] Completed [75/438] on J2 in 3.00s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNorms

   [junit4] Completed [76/438] on J2 in 6.86s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.TestCodecLoadingDeadlock

   [junit4] Completed [77/438] on J2 in 1.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDocBoost

   [junit4] Completed [78/438] on J2 in 0.08s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:03:25, stalled for  123s at: TestControlledRealTimeReopenThread.testControlledRealTimeReopenThread

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:03:49, stalled for 60.1s at: TestDemoParallelLeafReader.testRandomMultipleSchemaGensSameField

   [junit4] Suite: org.apache.lucene.util.TestOfflineSorter

   [junit4] Completed [79/438] on J2 in 49.20s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanNearQuery

   [junit4] Completed [80/438] on J2 in 0.04s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMaxTermFrequency

   [junit4] Completed [81/438] on J2 in 1.10s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexFileDeleter

   [junit4] Completed [82/438] on J2 in 1.57s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesCheckIndexHeader

   [junit4] Completed [83/438] on J2 in 3.59s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMergePolicyWrapper

   [junit4] Completed [84/438] on J2 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestUnicodeUtil

   [junit4] Completed [85/438] on J2 in 2.38s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelReaderEmptyIndex

   [junit4] Completed [86/438] on J2 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTieredMergePolicy

   [junit4] Completed [87/438] on J2 in 13.39s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestMergedIterator

   [junit4] Completed [88/438] on J2 in 0.63s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestIOUtils

   [junit4] Completed [89/438] on J2 in 0.16s, 18 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestExceedMaxTermLength

   [junit4] Completed [90/438] on J2 in 0.05s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:04:25, stalled for  183s at: TestControlledRealTimeReopenThread.testControlledRealTimeReopenThread

   [junit4] Suite: org.apache.lucene.search.TestBooleanMinShouldMatch

   [junit4] Completed [91/438] on J2 in 5.66s, 17 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestReaderWrapperDVTypeCheck

   [junit4] Completed [92/438] on J2 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCodecs

   [junit4] Completed [93/438] on J2 in 1.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSortedNumericSortField

   [junit4] Completed [94/438] on J2 in 0.20s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCheckIndex

   [junit4] Completed [95/438] on J2 in 1.02s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDocValuesRewriteMethod

   [junit4] Completed [96/438] on J2 in 8.36s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestAutomatonQueryUnicode

   [junit4] Completed [97/438] on J2 in 0.05s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestLazyProxSkipping

   [junit4] Completed [98/438] on J2 in 0.65s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestConsistentFieldNumbers

   [junit4] Completed [99/438] on J2 in 8.09s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexInput

   [junit4] Completed [100/438] on J2 in 1.57s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSubScorerFreqs

   [junit4] Completed [101/438] on J2 in 0.10s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestMultiMMap

   [junit4] Completed [102/438] on J2 in 15.88s, 56 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:05:13, stalled for 63.0s at: TestDemoParallelLeafReader.testRandomMultipleSchemaGens

   [junit4] Suite: org.apache.lucene.index.TestDemoParallelLeafReader

   [junit4] Completed [103/438] on J0 in 203.17s, 5 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:05:25, stalled for  243s at: TestControlledRealTimeReopenThread.testControlledRealTimeReopenThread

   [junit4] Suite: org.apache.lucene.search.TestNumericRangeQuery64

   [junit4] Completed [104/438] on J0 in 23.78s, 29 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFilterWeight

   [junit4] Completed [105/438] on J0 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNoDeletionPolicy

   [junit4] Completed [106/438] on J0 in 0.18s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocValuesIndexing

   [junit4] Completed [107/438] on J0 in 0.62s, 34 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestNumericRangeQuery32

   [junit4] Completed [108/438] on J2 in 52.83s, 23 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressDeletes

   [junit4] Completed [109/438] on J2 in 2.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBlendedTermQuery

   [junit4] Completed [110/438] on J2 in 0.08s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanBoostQuery

   [junit4] Completed [111/438] on J2 in 0.05s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestNativeFSLockFactory

   [junit4] Completed [112/438] on J2 in 9.87s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestNot

   [junit4] Completed [113/438] on J2 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMaxDocs

   [junit4] IGNOR/A 0.00s J2 | TestIndexWriterMaxDocs.testExactlyAtTrueLimit

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes over two hours))

   [junit4] Completed [114/438] on J2 in 12.49s, 21 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexableField

   [junit4] Completed [115/438] on J2 in 0.99s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:06:25, stalled for  303s at: TestControlledRealTimeReopenThread.testControlledRealTimeReopenThread

   [junit4] Suite: org.apache.lucene.search.TestControlledRealTimeReopenThread

   [junit4] IGNOR/A 0.00s J1 | TestControlledRealTimeReopenThread.testCRTReopen

   [junit4]    > Assumption #1: 'awaitsfix' test group is disabled (@AwaitsFix(bugUrl=https://issues.apache.org/jira/browse/LUCENE-5737))

   [junit4] Completed [116/438] on J1 in 316.73s, 6 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressAdvance

   [junit4] Completed [117/438] on J1 in 2.98s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexingSequenceNumbers

   [junit4] Completed [118/438] on J0 in 111.64s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestNearSpansOrdered

   [junit4] Completed [119/438] on J0 in 0.29s, 19 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.Test2BPagedBytes

   [junit4] IGNOR/A 0.00s J0 | Test2BPagedBytes.test

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=You must increase heap to > 2 G to run this))

   [junit4] Completed [120/438] on J0 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanationsWithFillerDocs

   [junit4] IGNOR/A 0.00s J0 | TestSimpleExplanationsWithFillerDocs.testMA2

   [junit4]    > Assumption #1: got: <[test is not viable with empty filler docs, null]>, expected: each not null

   [junit4] IGNOR/A 0.00s J0 | TestSimpleExplanationsWithFillerDocs.testMA1

   [junit4]    > Assumption #1: got: <[test is not viable with empty filler docs, null]>, expected: each not null

   [junit4] Completed [121/438] on J0 in 5.78s, 70 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanCoord

   [junit4] Completed [122/438] on J0 in 0.16s, 76 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestLiveFieldValues

   [junit4] Completed [123/438] on J0 in 1.52s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpansEnum

   [junit4] Completed [124/438] on J0 in 0.06s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestBytesRef

   [junit4] Completed [125/438] on J0 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanations

   [junit4] Completed [126/438] on J0 in 0.99s, 70 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestManyFields

   [junit4] Completed [127/438] on J0 in 0.91s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNeverDelete

   [junit4] Completed [128/438] on J0 in 5.24s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:07:49, stalled for 61.0s at: TestGrowableByteArrayDataOutput.testWriteLargeStrings

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestGrowableByteArrayDataOutput

   [junit4] Completed [129/438] on J1 in 80.16s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestRegexpQuery

   [junit4] Completed [130/438] on J1 in 0.17s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSloppyMath

   [junit4] Completed [131/438] on J1 in 1.78s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestSimpleFSDirectory

   [junit4] Completed [132/438] on J1 in 0.88s, 44 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestReadOnlyIndex

   [junit4] Completed [133/438] on J1 in 0.28s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterLockRelease

   [junit4] Completed [134/438] on J1 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BNumericDocValues

   [junit4] IGNOR/A 0.00s J1 | Test2BNumericDocValues.testNumerics

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 2 hours if the heap is 5gb))

   [junit4] Completed [135/438] on J1 in 0.01s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSameTokenSamePosition

   [junit4] Completed [136/438] on J1 in 0.03s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestAutomaton

   [junit4] Completed [137/438] on J1 in 13.00s, 56 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestThreadedForceMerge

   [junit4] Completed [138/438] on J1 in 0.79s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDocValuesScoring

   [junit4] Completed [139/438] on J1 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSetOnce

   [junit4] Completed [140/438] on J1 in 0.02s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestPagedBytes

   [junit4] Completed [141/438] on J1 in 2.44s, 4 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexCommit

   [junit4] Completed [142/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiset

   [junit4] Completed [143/438] on J1 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTwoPhaseCommitTool

   [junit4] Completed [144/438] on J1 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDoc

   [junit4] Completed [145/438] on J1 in 0.12s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestFilterSpans

   [junit4] Completed [146/438] on J1 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestBinaryTerms

   [junit4] Completed [147/438] on J1 in 0.26s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom

   [junit4] Completed [148/438] on J1 in 0.16s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:08:25, stalled for 63.2s at: TestLucene53NormsFormat.testNCommonBig

   [junit4] Suite: org.apache.lucene.search.TestMinShouldMatch2

   [junit4] Completed [149/438] on J1 in 4.42s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelTermEnum

   [junit4] Completed [150/438] on J1 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterDeleteQueue

   [junit4] Completed [151/438] on J1 in 0.11s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOutOfFileDescriptors

   [junit4] Completed [152/438] on J1 in 7.44s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanContainQuery

   [junit4] Completed [153/438] on J1 in 0.16s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCharsRefBuilder

   [junit4] Completed [154/438] on J1 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSortRescorer

   [junit4] Completed [155/438] on J1 in 0.95s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat

   [junit4] IGNOR/A 0.00s J2 | TestLucene53NormsFormat.testMergeStability

   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it

   [junit4] Completed [156/438] on J2 in 153.48s, 17 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestWordlistLoader

   [junit4] Completed [157/438] on J2 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiThreadTermVectors

   [junit4] Completed [158/438] on J2 in 3.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiValuedNumericRangeQuery

   [junit4] Completed [159/438] on J2 in 2.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBoolean2

   [junit4] Completed [160/438] on J1 in 39.77s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMerging

   [junit4] Completed [161/438] on J2 in 14.04s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiTermConstantScore

   [junit4] Completed [162/438] on J1 in 3.40s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestLockFactory

   [junit4] Completed [163/438] on J1 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFieldValueQuery

   [junit4] Completed [164/438] on J2 in 6.04s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanFirstQuery

   [junit4] Completed [165/438] on J2 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOrdinalMap

   [junit4] Completed [166/438] on J2 in 1.05s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressIndexing2

   [junit4] Completed [167/438] on J2 in 3.20s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene62.TestLucene62SegmentInfoFormat

   [junit4] Completed [168/438] on J2 in 0.48s, 16 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimilarityProvider

   [junit4] Completed [169/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFixedBitSet

   [junit4] Completed [170/438] on J1 in 22.30s, 19 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSortRandom

   [junit4] Completed [171/438] on J1 in 1.63s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMaxPosition

   [junit4] Completed [172/438] on J1 in 0.07s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelLeafReader

   [junit4] Completed [173/438] on J1 in 0.51s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormat

   [junit4] Completed [174/438] on J2 in 19.57s, 17 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:09:50, stalled for 69.2s at: TestLucene54DocValuesFormat.testSortedSetVariableLengthBigVsStoredFields

   [junit4] Suite: org.apache.lucene.codecs.lucene60.TestLucene60PointsFormat

   [junit4] IGNOR/A 0.01s J1 | TestLucene60PointsFormat.testMergeStability

   [junit4]    > Assumption #1: merge is not stable

   [junit4] Completed [175/438] on J1 in 36.33s, 17 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastCompressionMode

   [junit4] Completed [176/438] on J1 in 1.07s, 11 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:10:49, stalled for 62.6s at: TestBagOfPostings.test

   [junit4] Suite: org.apache.lucene.index.TestBagOfPostings

   [junit4] Completed [177/438] on J2 in 65.67s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiCollector

   [junit4] Completed [178/438] on J2 in 0.67s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharacterUtils

   [junit4] Completed [179/438] on J2 in 0.01s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestReqExclBulkScorer

   [junit4] Completed [180/438] on J2 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestField

   [junit4] Completed [181/438] on J2 in 0.04s, 27 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPayloadsOnVectors

   [junit4] Completed [182/438] on J2 in 0.04s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFilterLeafReader

   [junit4] Completed [183/438] on J2 in 0.02s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.fst.TestBytesStore

   [junit4] Completed [184/438] on J2 in 8.60s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSearchWithThreads

   [junit4] Completed [185/438] on J1 in 53.54s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleSearchEquivalence

   [junit4] Completed [186/438] on J1 in 7.57s, 17 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.packed.TestPackedInts

   [junit4] Completed [187/438] on J2 in 29.98s, 28 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAddIndexes

   [junit4] Completed [188/438] on J2 in 6.85s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMixedDocValuesUpdates

   [junit4] Completed [189/438] on J1 in 31.32s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSameScoresWithThreads

   [junit4] Completed [190/438] on J1 in 1.16s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestNIOFSDirectory

   [junit4] Completed [191/438] on J1 in 2.38s, 44 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermdocPerf

   [junit4] Completed [192/438] on J1 in 0.03s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanQuery

   [junit4] Completed [193/438] on J1 in 0.74s, 22 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestShardSearching

   [junit4] Completed [194/438] on J2 in 19.66s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:13:02, stalled for 69.3s at: TestLucene54DocValuesFormat.testSparseDocValuesVsStoredFields

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:13:25, stalled for 63.5s at: TestIndexWriterExceptions.testTooManyTokens

   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat

   [junit4] Completed [195/438] on J1 in 97.41s, 96 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestUsageTrackingFilterCachingPolicy

   [junit4] Completed [196/438] on J1 in 0.01s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:14:02, stalled for  129s at: TestLucene54DocValuesFormat.testSparseDocValuesVsStoredFields

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50TermVectorsFormat

   [junit4] Completed [197/438] on J1 in 48.12s, 17 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveChecksumFooter

   [junit4] Completed [198/438] on J1 in 0.17s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:14:25, stalled for  124s at: TestIndexWriterExceptions.testTooManyTokens

   [junit4] Suite: org.apache.lucene.search.TestScorerPerf

   [junit4] Completed [199/438] on J1 in 7.68s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestStopFilter

   [junit4] Completed [200/438] on J1 in 0.02s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTopDocsCollector

   [junit4] Completed [201/438] on J1 in 0.15s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat

   [junit4] IGNOR/A 0.00s J1 | TestPerFieldPostingsFormat.testPostingsEnumReuse

   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it

   [junit4] IGNOR/A 0.00s J1 | TestPerFieldPostingsFormat.testMergeStability

   [junit4]    > Assumption #1: The MockRandom PF randomizes content on the fly, so we can't check it

   [junit4] Completed [202/438] on J1 in 25.50s, 24 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTryDelete

   [junit4] Completed [203/438] on J1 in 0.04s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterCommit

   [junit4] Completed [204/438] on J1 in 2.36s, 14 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:15:02, stalled for  189s at: TestLucene54DocValuesFormat.testSparseDocValuesVsStoredFields

   [junit4] Suite: org.apache.lucene.index.TestDeletionPolicy

   [junit4] Completed [205/438] on J1 in 5.08s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterExceptions

   [junit4] Completed [206/438] on J2 in 205.28s, 36 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFlushByRamOrCountsPolicy

   [junit4] Completed [207/438] on J2 in 1.87s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterDelete

   [junit4] Completed [208/438] on J2 in 26.83s, 27 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterMergePolicy

   [junit4] Completed [209/438] on J2 in 3.93s, 7 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:16:02, stalled for  249s at: TestLucene54DocValuesFormat.testSparseDocValuesVsStoredFields

   [junit4] Suite: org.apache.lucene.search.TestBooleanOr

   [junit4] Completed [210/438] on J1 in 63.17s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecs

   [junit4] Completed [211/438] on J1 in 19.55s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDirectoryReaderReopen

   [junit4] Completed [212/438] on J1 in 2.40s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTimeLimitingCollector

   [junit4] Completed [213/438] on J1 in 6.37s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestHighCompressionMode

   [junit4] Completed [214/438] on J1 in 1.99s, 7 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:17:02, stalled for 63.7s at: TestNRTThreads.testNRTThreads

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:17:02, stalled for  309s at: TestLucene54DocValuesFormat.testSparseDocValuesVsStoredFields

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:17:50, stalled for 71.4s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:18:02, stalled for  124s at: TestNRTThreads.testNRTThreads

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:18:50, stalled for  131s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:19:02, stalled for  184s at: TestNRTThreads.testNRTThreads

   [junit4] Suite: org.apache.lucene.codecs.lucene54.TestLucene54DocValuesFormat

   [junit4] Completed [215/438] on J0 in 692.42s, 109 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestCharTermAttributeImpl

   [junit4] Completed [216/438] on J0 in 0.45s, 12 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSearchAfter

   [junit4] Completed [217/438] on J0 in 1.84s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:19:50, stalled for  192s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:20:02, stalled for  244s at: TestNRTThreads.testNRTThreads

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:20:26, stalled for 66.2s at: TestSearcherManager.testSearcherManager

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:20:50, stalled for  252s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:21:02, stalled for  304s at: TestNRTThreads.testNRTThreads

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:21:26, stalled for  126s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.index.TestNRTThreads

   [junit4] Completed [218/438] on J2 in 332.43s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:21:50, stalled for  312s at: TestDocumentsWriterStallControl.testAccquireReleaseRace

   [junit4] Suite: org.apache.lucene.search.TestTopDocsMerge

   [junit4] Completed [219/438] on J2 in 24.87s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestLongPostings

   [junit4] Completed [220/438] on J2 in 4.03s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressIndexing

   [junit4] Completed [221/438] on J2 in 5.00s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSnapshotDeletionPolicy

   [junit4] Completed [222/438] on J2 in 1.43s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterForceMerge

   [junit4] Completed [223/438] on J2 in 3.91s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestArrayUtil

   [junit4] Completed [224/438] on J2 in 5.88s, 14 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestLevenshteinAutomata

   [junit4] Completed [225/438] on J2 in 1.77s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestByteSlices

   [junit4] Completed [226/438] on J2 in 7.28s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:22:26, stalled for  186s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.store.TestBufferedIndexInput

   [junit4] Completed [227/438] on J2 in 0.78s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocumentsWriterStallControl

   [junit4] Completed [228/438] on J1 in 365.81s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDirectoryReader

   [junit4] Completed [229/438] on J1 in 3.15s, 24 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentMerger

   [junit4] Completed [230/438] on J1 in 0.80s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPhraseQuery

   [junit4] Completed [231/438] on J1 in 1.66s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestWeakIdentityMap

   [junit4] Completed [232/438] on J2 in 23.16s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCollectionUtil

   [junit4] Completed [233/438] on J1 in 16.40s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOmitNorms

   [junit4] Completed [234/438] on J1 in 2.41s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestRegexpRandom2

   [junit4] Completed [235/438] on J1 in 5.48s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestNRTCachingDirectory

   [junit4] IGNOR/A 0.00s J1 | TestNRTCachingDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] IGNOR/A 0.00s J1 | TestNRTCachingDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] Completed [236/438] on J1 in 2.70s, 46 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentReader

   [junit4] Completed [237/438] on J1 in 1.08s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.compressing.TestFastDecompressionMode

   [junit4] Completed [238/438] on J1 in 3.35s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestParallelCompositeReader

   [junit4] Completed [239/438] on J1 in 0.84s, 15 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:23:26, stalled for  246s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterNRTIsCurrent

   [junit4] Completed [240/438] on J1 in 3.25s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTransactions

   [junit4] Completed [241/438] on J1 in 2.52s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.perfield.TestPerFieldPostingsFormat2

   [junit4] Completed [242/438] on J1 in 5.02s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSort

   [junit4] Completed [243/438] on J1 in 0.41s, 25 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestBagOfPositions

   [junit4] Completed [244/438] on J2 in 46.67s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestToken

   [junit4] Completed [245/438] on J2 in 0.11s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestCustomSearcherSort

   [junit4] Completed [246/438] on J2 in 8.31s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiDocValues

   [junit4] Completed [247/438] on J1 in 11.33s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.similarities.TestSimilarityBase

   [junit4] Completed [248/438] on J2 in 1.58s, 32 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestLegacyNumericUtils

   [junit4] Completed [249/438] on J1 in 2.16s, 12 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminizeLexicon

   [junit4] Completed [250/438] on J2 in 3.49s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpans

   [junit4] Completed [251/438] on J2 in 0.81s, 26 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPostingsOffsets

   [junit4] Completed [252/438] on J2 in 3.05s, 13 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMixedCodecs

   [junit4] Completed [253/438] on J2 in 4.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTransactionRollback

   [junit4] Completed [254/438] on J2 in 0.48s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFlex

   [junit4] Completed [255/438] on J2 in 0.60s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanations

   [junit4] Completed [256/438] on J2 in 0.60s, 30 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestRAMDirectory

   [junit4] IGNOR/A 0.00s J2 | TestRAMDirectory.testPendingDeletions

   [junit4]    > Assumption #1: we can only install VirusCheckingFS on an FSDirectory

   [junit4] IGNOR/A 0.01s J2 | TestRAMDirectory.testFsyncDoesntCreateNewFiles

   [junit4]    > Assumption #1: test only works for FSDirectory subclasses

   [junit4] Completed [257/438] on J2 in 1.37s, 48 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiPhraseQuery

   [junit4] Completed [258/438] on J2 in 0.41s, 17 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestDeterminism

   [junit4] Completed [259/438] on J2 in 1.94s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocsAndPositions

   [junit4] Completed [260/438] on J2 in 1.04s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestMergeSchedulerExternal

   [junit4] Completed [261/438] on J2 in 0.58s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermVectorsWriter

   [junit4] Completed [262/438] on J2 in 0.33s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOmitPositions

   [junit4] Completed [263/438] on J2 in 0.79s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestBytesRefHash

   [junit4] Completed [264/438] on J2 in 3.42s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnDiskFull

   [junit4] Completed [265/438] on J1 in 35.36s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestDisjunctionMaxQuery

   [junit4] Completed [266/438] on J1 in 0.25s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestForceMergeForever

   [junit4] Completed [267/438] on J1 in 0.97s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestOmitTf

   [junit4] Completed [268/438] on J1 in 0.49s, 6 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:24:26, stalled for  306s at: TestSearcherManager.testSearcherManager

   [junit4] Suite: org.apache.lucene.index.TestMultiFields

   [junit4] Completed [269/438] on J1 in 1.49s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTermRangeQuery

   [junit4] Completed [270/438] on J1 in 0.21s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCrashCausesCorruptIndex

   [junit4] Completed [271/438] on J1 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestForTooMuchCloning

   [junit4] Completed [272/438] on J1 in 0.08s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestFuzzyQuery

   [junit4] Completed [273/438] on J1 in 0.09s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPayloads

   [junit4] Completed [274/438] on J1 in 0.35s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestWildcard

   [junit4] Completed [275/438] on J1 in 0.07s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestFieldMaskingSpanQuery

   [junit4] IGNOR/A 0.00s J1 | TestFieldMaskingSpanQuery.testSimple2

   [junit4]    > Assumption #1: Broken scoring: LUCENE-3723

   [junit4] IGNOR/A 0.00s J1 | TestFieldMaskingSpanQuery.testSpans2

   [junit4]    > Assumption #1: Broken scoring: LUCENE-3723

   [junit4] Completed [276/438] on J1 in 0.22s, 11 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestExternalCodecs

   [junit4] Completed [277/438] on J1 in 0.83s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimpleExplanationsOfNonMatches

   [junit4] Completed [278/438] on J1 in 0.05s, 70 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestTermVectorsReader

   [junit4] Completed [279/438] on J1 in 0.13s, 13 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPerSegmentDeletes

   [junit4] Completed [280/438] on J1 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPrefixInBooleanQuery

   [junit4] Completed [281/438] on J1 in 0.36s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentTermEnum

   [junit4] Completed [282/438] on J1 in 0.11s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestUniqueTermCount

   [junit4] Completed [283/438] on J1 in 0.04s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestComplexExplanationsOfNonMatches

   [junit4] Completed [284/438] on J1 in 0.14s, 21 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestPrefixCodedTerms

   [junit4] Completed [285/438] on J1 in 0.35s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestAllFilesHaveCodecHeader

   [junit4] Completed [286/438] on J1 in 0.46s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestAutomatonQuery

   [junit4] Completed [287/438] on J1 in 1.47s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestDocument

   [junit4] Completed [288/438] on J1 in 0.16s, 12 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTopFieldCollector

   [junit4] Completed [289/438] on J1 in 0.13s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestWildcardRandom

   [junit4] Completed [290/438] on J1 in 0.38s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPrefixRandom

   [junit4] Completed [291/438] on J1 in 0.79s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiLevelSkipList

   [junit4] Completed [292/438] on J1 in 0.08s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSloppyPhraseQuery2

   [junit4] Completed [293/438] on J2 in 23.61s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanMultiTermQueryWrapper

   [junit4] Completed [294/438] on J1 in 0.07s, 9 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestSearchForDuplicates

   [junit4] Completed [295/438] on J2 in 0.15s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSizeBoundedForceMerge

   [junit4] Completed [296/438] on J1 in 0.11s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestConstantScoreQuery

   [junit4] Completed [297/438] on J1 in 0.07s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.automaton.TestCompiledAutomaton

   [junit4] Completed [298/438] on J2 in 0.33s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestElevationComparator

   [junit4] Completed [299/438] on J1 in 0.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterConfig

   [junit4] Completed [300/438] on J2 in 0.03s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestConjunctions

   [junit4] Completed [301/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestMultiTermQueryRewrites

   [junit4] Completed [302/438] on J2 in 0.03s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPositionIncrement

   [junit4] Completed [303/438] on J1 in 0.06s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSimilarity

   [junit4] Completed [304/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.spans.TestSpanExplanationsOfNonMatches

   [junit4] Completed [305/438] on J1 in 0.04s, 30 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestVirtualMethod

   [junit4] Completed [306/438] on J2 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRollingBuffer

   [junit4] Completed [307/438] on J1 in 0.10s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestSmallFloat

   [junit4] Completed [308/438] on J2 in 0.12s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestNumericTokenStream

   [junit4] Completed [309/438] on J1 in 0.03s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestSearch

   [junit4] Completed [310/438] on J2 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCachingTokenFilter

   [junit4] Completed [311/438] on J1 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestFilterIterator

   [junit4] Completed [312/438] on J2 in 0.02s, 8 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestBinaryDocument

   [junit4] Completed [313/438] on J2 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestAttributeSource

   [junit4] Completed [314/438] on J2 in 0.01s, 7 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRecyclingByteBlockAllocator

   [junit4] Completed [315/438] on J2 in 0.03s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestRamUsageEstimator

   [junit4] IGNOR/A 0.00s J2 | TestRamUsageEstimator.testPrintValues

   [junit4]    > Assumption #1: Specify -Dtests.verbose=true to print constants of RamUsageEstimator.

   [junit4] Completed [316/438] on J2 in 0.01s, 5 tests, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIsCurrent

   [junit4] Completed [317/438] on J2 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNoMergeScheduler

   [junit4] Completed [318/438] on J2 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestNamedSPILoader

   [junit4] Completed [319/438] on J2 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPhrasePrefixQuery

   [junit4] Completed [320/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestByteBlockPool

   [junit4] Completed [321/438] on J2 in 0.10s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestDemo

   [junit4] Completed [322/438] on J2 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.TestAssertions

   [junit4] Completed [323/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharFilter

   [junit4] Completed [324/438] on J2 in 0.01s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestRollback

   [junit4] Completed [325/438] on J2 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestCloseableThreadLocal

   [junit4] Completed [326/438] on J2 in 0.01s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterOnJRECrash

   [junit4] Completed [327/438] on J2 in 4.85s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestSearcherManager

   [junit4] Completed [328/438] on J0 in 340.78s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMergeRateLimiter

   [junit4] Completed [329/438] on J0 in 0.03s, 1 test

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:25:50, stalled for 61.4s at: Test4GBStoredFields.test

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:25:50, stalled for 70.1s at: Test2BPostings.test

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:26:02, stalled for 61.6s at: TestPointQueries.testRandomBinaryBig

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:26:50, stalled for  130s at: Test2BPostings.test

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:26:50, stalled for  121s at: Test4GBStoredFields.test

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:27:50, stalled for  190s at: Test2BPostings.test

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:27:50, stalled for  181s at: Test4GBStoredFields.test

   [junit4] Suite: org.apache.lucene.search.TestPointQueries

   [junit4] Completed [330/438] on J0 in 189.68s, 46 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestDirectory

   [junit4] Completed [331/438] on J0 in 0.40s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestStressNRT

   [junit4] Completed [332/438] on J0 in 0.22s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestBooleanScorer

   [junit4] Completed [333/438] on J0 in 12.06s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestDateTools

   [junit4] Completed [334/438] on J0 in 0.01s, 5 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTermScorer

   [junit4] Completed [335/438] on J0 in 0.06s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50StoredFieldsFormatHighCompression

   [junit4] Completed [336/438] on J0 in 26.13s, 19 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:28:50, stalled for  241s at: Test4GBStoredFields.test

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:28:50, stalled for  250s at: Test2BPostings.test

   [junit4] Suite: org.apache.lucene.index.TestCustomNorms

   [junit4] Completed [337/438] on J0 in 2.41s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPrefixQuery

   [junit4] Completed [338/438] on J0 in 14.30s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.standard.TestStandardAnalyzer

   [junit4] Completed [339/438] on J0 in 6.88s, 34 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestTermQuery

   [junit4] Completed [340/438] on J0 in 0.07s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.document.TestFieldType

   [junit4] Completed [341/438] on J0 in 0.02s, 3 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:29:50, stalled for  301s at: Test4GBStoredFields.test

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:29:50, stalled for  310s at: Test2BPostings.test

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:30:26, stalled for 62.6s at: TestTermsEnum.testVaryingTermsPerSegment

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:30:50, stalled for  361s at: Test4GBStoredFields.test

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:30:50, stalled for  370s at: Test2BPostings.test

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:31:26, stalled for  123s at: TestTermsEnum.testVaryingTermsPerSegment

   [junit4] Suite: org.apache.lucene.index.Test2BPostings

   [junit4] Completed [342/438] on J1 in 423.65s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestCodecHoldsOpenFiles

   [junit4] Completed [343/438] on J1 in 0.11s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestReaderClosed

   [junit4] Completed [344/438] on J1 in 0.03s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestScoreCachingWrappingScorer

   [junit4] Completed [345/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.util.TestIntsRef

   [junit4] Completed [346/438] on J1 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPositions

   [junit4] IGNOR/A 0.00s J1 | Test2BPositions.test

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=uses lots of space and takes a few minutes))

   [junit4] Completed [347/438] on J1 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPostingsBytes

   [junit4] IGNOR/A 0.00s J1 | Test2BPostingsBytes.test

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~20GB-30GB of space and 10 minutes))

   [junit4] Completed [348/438] on J1 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.search.TestPositiveScoresOnlyCollector

   [junit4] Completed [349/438] on J1 in 0.06s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BTerms

   [junit4] IGNOR/A 0.00s J1 | Test2BTerms.test2BTerms

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=very slow, use 5g minimum heap))

   [junit4] Completed [350/438] on J1 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestNewestSegment

   [junit4] Completed [351/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.store.TestByteArrayDataInput

   [junit4] Completed [352/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BBinaryDocValues

   [junit4] IGNOR/A 0.00s J1 | Test2BBinaryDocValues.testVariableBinary

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))

   [junit4] IGNOR/A 0.00s J1 | Test2BBinaryDocValues.testFixedBinary

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes ~ 6 hours if the heap is 5gb))

   [junit4] Completed [353/438] on J1 in 0.00s, 2 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharArrayMap

   [junit4] Completed [354/438] on J1 in 0.09s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestCharArraySet

   [junit4] Completed [355/438] on J1 in 0.03s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.TestReusableStringReader

   [junit4] Completed [356/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestBytesRefAttImpl

   [junit4] Completed [357/438] on J1 in 0.01s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.analysis.tokenattributes.TestPackedTokenAttributeImpl

   [junit4] Completed [358/438] on J1 in 0.03s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.TestCodecUtil

   [junit4] Completed [359/438] on J1 in 0.03s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test4GBStoredFields

   [junit4] Completed [360/438] on J2 in 420.67s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2

   [junit4] Completed [361/438] on J2 in 1.17s, 4 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat3

   [junit4] Completed [362/438] on J2 in 4.99s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestForUtil

   [junit4] Completed [363/438] on J2 in 0.07s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50CompoundFormat

   [junit4] Completed [364/438] on J2 in 3.63s, 26 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestLucene50FieldInfoFormat

   [junit4] Completed [365/438] on J2 in 0.34s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestGeoEncodingUtils

   [junit4] Completed [366/438] on J2 in 0.78s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat

   [junit4] Completed [367/438] on J1 in 29.73s, 26 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestPolygon

   [junit4] Completed [368/438] on J1 in 0.03s, 15 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestGeoUtils

   [junit4] Completed [369/438] on J2 in 19.20s, 6 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BPoints

   [junit4] IGNOR/A 0.00s J2 | Test2BPoints.test2D

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))

   [junit4] IGNOR/A 0.00s J2 | Test2BPoints.test1D

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=takes at least 4 hours and consumes many GB of temp disk space))

   [junit4] Completed [370/438] on J2 in 0.01s, 2 tests, 2 skipped

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.Test2BSortedDocValuesFixedSorted

   [junit4] IGNOR/A 0.00s J2 | Test2BSortedDocValuesFixedSorted.testFixedSorted

   [junit4]    > Assumption #1: 'monster' test group is disabled (@Monster(value=Takes ~ 6 hours if the heap is 5gb))

   [junit4] Completed [371/438] on J2 in 0.00s, 1 test, 1 skipped

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:32:26, stalled for  183s at: TestTermsEnum.testVaryingTermsPerSegment

   [junit4] Suite: org.apache.lucene.index.TestTermsEnum

   [junit4] Completed [372/438] on J0 in 195.21s, 11 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterExceptions2

   [junit4] Completed [373/438] on J0 in 10.36s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestDocInverterPerFieldErrorInfo

   [junit4] Completed [374/438] on J0 in 0.07s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.geo.TestPolygon2D

   [junit4] Completed [375/438] on J1 in 26.58s, 10 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestExitableDirectoryReader

   [junit4] Completed [376/438] on J1 in 0.67s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestFilterDirectoryReader

   [junit4] Completed [377/438] on J1 in 0.05s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexReaderClose

   [junit4] Completed [378/438] on J1 in 1.02s, 2 tests

   [junit4] 

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:33:38, stalled for 69.4s at: TestBinaryDocValuesUpdates.testTonsOfUpdates

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:33:50, stalled for 71.4s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:34:14, stalled for 62.8s at: TestIndexSorting.testRandom3

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:34:38, stalled for  129s at: TestBinaryDocValuesUpdates.testTonsOfUpdates

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:34:50, stalled for  131s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:35:14, stalled for  123s at: TestIndexSorting.testRandom3

   [junit4] Suite: org.apache.lucene.index.TestBinaryDocValuesUpdates

   [junit4] Completed [379/438] on J2 in 178.44s, 28 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestIndexWriterFromReader

   [junit4] Completed [380/438] on J2 in 0.95s, 12 tests

   [junit4] 

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:35:50, stalled for  192s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:36:14, stalled for  183s at: TestIndexSorting.testRandom3

   [junit4] HEARTBEAT J2 PID(30170@localhost): 2016-07-30T08:36:26, stalled for 61.5s at: TestIndexWriterOnVMError.testCheckpoint

   [junit4] HEARTBEAT J0 PID(30154@localhost): 2016-07-30T08:36:50, stalled for  252s at: TestDuelingCodecsAtNight.testBigEquals

   [junit4] HEARTBEAT J1 PID(30169@localhost): 2016-07-30T08:37:14, stalled for  243s at: TestIndexSorting.testRandom3

   [junit4] Suite: org.apache.lucene.index.TestDuelingCodecsAtNight

   [junit4] Completed [381/438] on J0 in 286.33s, 3 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestInfoStream

   [junit4] Completed [382/438] on J0 in 0.01s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestLogMergePolicy

   [junit4] Completed [383/438] on J0 in 0.02s, 2 tests

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestMultiTermsEnum

   [junit4] Completed [384/438] on J0 in 0.02s, 1 test

   [junit4] 

   [junit4] Suite: org.apache.lucene.index.TestSegmentInfos

   [junit4] Completed [385/438] on J0 in 0.01s, 3 tests

   [junit4] 

   [junit4

[...truncated too long message...]

sts-6.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/bcprov-jdk15-1.45.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/cglib-nodep-2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/commons-math3-3.4.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/easymock-3.0.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-common-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-hdfs-2.7.2-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/hadoop-minikdc-2.7.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-annotations-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-core-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-databind-2.5.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jackson-mapper-asl-1.9.13.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/mockito-core-1.9.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/objenesis-1.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/presto-parser-0.122.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/core/test-lib/slice-0.10.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/analysis/icu/lucene-analyzers-icu-6.2.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/contrib/analysis-extras/lib/icu4j-56.1.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.3.0.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-antlr.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-regexp.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-resolver.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-net.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jmf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-oro.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-xalan2.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-commons-logging.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jsch.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit4.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-javamail.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jai.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-jdepend.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-junit.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bsf.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-swing.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-netrexx.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-bcel.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-testutil.jar:/home/jenkins/jenkins-slave/tools/hudson.tasks.Ant_AntInstallation/ant-1.8.2/lib/ant-apache-log4j.jar:/x1/jenkins/jenkins-slave/tools/hudson.model.JDK/latest1.8/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.3.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160730_103644_416.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build/solr-core/test/temp/junit4-J0-20160730_103644_416.suites -stdin
   [junit4] ERROR: JVM J0 ended with an exception: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1501)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:121)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:955)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:952)
   [junit4] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   [junit4] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4] 	at java.lang.Thread.run(Thread.java:745)

BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:771: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:707: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/build.xml:233: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/solr/common-build.xml:536: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:1443: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-6.x/lucene/common-build.xml:984: At least one slave process threw an exception, first: Quit event not received from the forked process? This may indicate JVM crash or runner bugs.

Total time: 264 minutes 0 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
ERROR: Failed to archive {heapdumps/java_pid1074.hprof=heapdumps/java_pid1074.hprof, README.txt=README.txt, solr/build/solr-core/test/temp/junit4-J1-20160730_103644_416.events=solr/build/solr-core/test/temp/junit4-J1-20160730_103644_416.events, solr/build/solr-core/test/temp/junit4-J2-20160730_103644_416.events=solr/build/solr-core/test/temp/junit4-J2-20160730_103644_416.events, solr/build/solr-core/test/temp/junit4-J0-20160730_103644_416.events=solr/build/solr-core/test/temp/junit4-J0-20160730_103644_416.events} due to internal error; falling back to full archiving
java.lang.NullPointerException
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any





[JENKINS] Lucene-Solr-NightlyTests-6.x - Build # 130 - Failure

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-6.x/130/

No tests ran.

Build Log:
[...truncated 27 lines...]
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any