You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2019/03/05 04:07:52 UTC

[JENKINS] Lucene-Solr-7.x-Windows (32bit/jdk1.8.0_172) - Build # 1014 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/1014/
Java: 32bit/jdk1.8.0_172 -server -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.core.TestNRTOpen

Error Message:
Could not remove the following files (in the order of attempts):    C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001\init-core-data-001: java.nio.file.AccessDeniedException: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001\init-core-data-001    C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of attempts):
   C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001\init-core-data-001: java.nio.file.AccessDeniedException: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001\init-core-data-001
   C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001

	at __randomizedtesting.SeedInfo.seed([83ECD4C20377A6F5]:0)
	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:318)
	at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13286 lines...]
   [junit4] Suite: org.apache.solr.core.TestNRTOpen
   [junit4]   2> 961112 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001\init-core-data-001
   [junit4]   2> 961114 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 961115 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 961116 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 961117 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/C:/Users/jenkins/workspace/Lucene-Solr-7.x-Windows/solr/core/src/test-files/solr/collection1/lib, /C:/Users/jenkins/workspace/Lucene-Solr-7.x-Windows/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 961171 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.8.0
   [junit4]   2> 961181 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=minimal
   [junit4]   2> 961188 WARN  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.s.IndexSchema no uniqueKey specified in schema.
   [junit4]   2> 961188 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field null
   [junit4]   2> 961627 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 961722 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 961722 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 961752 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bfe1b
   [junit4]   2> 961754 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bfe1b
   [junit4]   2> 961757 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bfe1b
   [junit4]   2> 961759 INFO  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/C:/Users/jenkins/workspace/Lucene-Solr-7.x-Windows/solr/core/src/test-files/solr/collection1/lib, /C:/Users/jenkins/workspace/Lucene-Solr-7.x-Windows/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 961843 INFO  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.8.0
   [junit4]   2> 961853 INFO  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=minimal
   [junit4]   2> 961857 WARN  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.s.IndexSchema no uniqueKey specified in schema.
   [junit4]   2> 961857 INFO  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field null
   [junit4]   2> 961857 INFO  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1, trusted=true
   [junit4]   2> 961857 INFO  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bfe1b
   [junit4]   2> 961857 INFO  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001\init-core-data-001\]
   [junit4]   2> 961997 INFO  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 962000 INFO  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 962001 INFO  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1a7687[collection1] main]
   [junit4]   2> 962001 WARN  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf; switching to use InMemory storage instead.
   [junit4]   2> 962001 INFO  (coreLoadExecutor-4265-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 962005 INFO  (searcherExecutor-4266-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1a7687[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 962006 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 962009 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[(null)]} 0 2
   [junit4]   2> 962016 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 962016 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1e1eac commitCommandVersion:0
   [junit4]   2> 962045 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@14c9ec0[collection1] main]
   [junit4]   2> 962045 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 962046 INFO  (searcherExecutor-4266-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@14c9ec0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753603315}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 962046 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 30
   [junit4]   2> 962048 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 962048 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=4720369
   [junit4]   2> 962049 INFO  (coreCloseExecutor-4270-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@19d125
   [junit4]   2> 962049 INFO  (coreCloseExecutor-4270-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=19d125
   [junit4]   2> 962049 INFO  (coreCloseExecutor-4270-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@c5b358: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@1b782f7
   [junit4]   2> 962074 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 962074 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@e60219: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1dee77
   [junit4]   2> 962077 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 962077 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7c96c9: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@16815aa
   [junit4]   2> 962081 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 962081 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1f46ff8: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1141728
   [junit4]   2> 962084 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 962086 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/C:/Users/jenkins/workspace/Lucene-Solr-7.x-Windows/solr/core/src/test-files/solr/collection1/lib, /C:/Users/jenkins/workspace/Lucene-Solr-7.x-Windows/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 962162 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.8.0
   [junit4]   2> 962180 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.s.IndexSchema [null] Schema name=minimal
   [junit4]   2> 962186 WARN  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.s.IndexSchema no uniqueKey specified in schema.
   [junit4]   2> 962187 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field null
   [junit4]   2> 962826 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 962902 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 962902 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 962930 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bfe1b
   [junit4]   2> 962935 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bfe1b
   [junit4]   2> 962935 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bfe1b
   [junit4]   2> 962937 INFO  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/C:/Users/jenkins/workspace/Lucene-Solr-7.x-Windows/solr/core/src/test-files/solr/collection1/lib, /C:/Users/jenkins/workspace/Lucene-Solr-7.x-Windows/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 963021 INFO  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.8.0
   [junit4]   2> 963033 INFO  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=minimal
   [junit4]   2> 963039 WARN  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.s.IndexSchema no uniqueKey specified in schema.
   [junit4]   2> 963039 INFO  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field null
   [junit4]   2> 963039 INFO  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1, trusted=true
   [junit4]   2> 963039 INFO  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bfe1b
   [junit4]   2> 963039 INFO  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001\init-core-data-001\]
   [junit4]   2> 963182 INFO  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 963182 INFO  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 963188 INFO  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@61bed8[collection1] main]
   [junit4]   2> 963188 WARN  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf; switching to use InMemory storage instead.
   [junit4]   2> 963188 INFO  (coreLoadExecutor-4280-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 963192 INFO  (searcherExecutor-4281-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@61bed8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.8.0):c1:[diagnostics={java.runtime.version=1.8.0_172-b11, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os=Windows 10, os.arch=x86, os.version=10.0, source=flush, timestamp=1551753603315}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 963193 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 963197 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testSharedCores
   [junit4]   2> 963197 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 963199 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{deleteByQuery=*:*} 0 0
   [junit4]   2> 963199 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[(null)]} 0 0
   [junit4]   2> 963199 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 963199 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@b2f6d5 commitCommandVersion:0
   [junit4]   2> 963210 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@14b0b94[collection1] main]
   [junit4]   2> 963211 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 963212 INFO  (searcherExecutor-4281-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@14b0b94[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604490}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 963212 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 12
   [junit4]   2> 963212 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=true,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 963212 WARN  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 Starting optimize... Reading and rewriting a potentially large percent of the entire index, reducing to 2147483647 segments
   [junit4]   2> 963213 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 963213 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 963218 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{optimize=} 0 5
   [junit4]   2> 963220 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[(null)]} 0 1
   [junit4]   2> 963221 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 963230 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@c632eb[collection1] main]
   [junit4]   2> 963230 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 963230 INFO  (searcherExecutor-4281-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@c632eb[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604490}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_2(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604512}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 963232 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 10
   [junit4]   2> 963233 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[(null)]} 0 1
   [junit4]   2> 963233 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[(null)]} 0 0
   [junit4]   2> 963233 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 963233 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@b2f6d5 commitCommandVersion:0
   [junit4]   2> 963246 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1f7dccf[collection1] main]
   [junit4]   2> 963246 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 963246 INFO  (searcherExecutor-4281-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1f7dccf[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604490}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_2(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604512}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_3(7.8.0):c2:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604524}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 963246 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 13
   [junit4]   2> 963248 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{deleteByQuery=foo2:doc} 0 0
   [junit4]   2> 963248 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 963248 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@b2f6d5 commitCommandVersion:0
   [junit4]   2> 963253 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@8e5033[collection1] main]
   [junit4]   2> 963253 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 963254 INFO  (searcherExecutor-4281-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@8e5033[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_1(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604490}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_2(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604512}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_3(7.8.0):c2/1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604524}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]:delGen=1)))}
   [junit4]   2> 963254 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 6
   [junit4]   2> 963255 INFO  (TEST-TestNRTOpen.testSharedCores-seed#[83ECD4C20377A6F5]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testSharedCores
   [junit4]   2> 963258 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testReaderIsNRT
   [junit4]   2> 963259 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 963259 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{deleteByQuery=*:*} 0 0
   [junit4]   2> 963260 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[(null)]} 0 1
   [junit4]   2> 963260 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 963260 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@b2f6d5 commitCommandVersion:0
   [junit4]   2> 963273 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1e72729[collection1] main]
   [junit4]   2> 963275 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 963276 INFO  (searcherExecutor-4281-thread-1) [    ] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1e72729[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604552}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 963278 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 17
   [junit4]   2> 963279 ERROR (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\core.properties:java.nio.file.NoSuchFileException: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\core.properties
   [junit4]   2> 963279 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/C:/Users/jenkins/workspace/Lucene-Solr-7.x-Windows/solr/core/src/test-files/solr/collection1/lib, /C:/Users/jenkins/workspace/Lucene-Solr-7.x-Windows/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 963364 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.8.0
   [junit4]   2> 963377 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.s.IndexSchema [collection1] Schema name=minimal
   [junit4]   2> 963382 WARN  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.s.IndexSchema no uniqueKey specified in schema.
   [junit4]   2> 963382 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field null
   [junit4]   2> 963382 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    ] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1
   [junit4]   2> 963414 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bfe1b
   [junit4]   2> 963414 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001\init-core-data-001\]
   [junit4]   2> 963532 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 963532 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 963533 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4732f0[collection1] main]
   [junit4]   2> 963534 WARN  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf; switching to use InMemory storage instead.
   [junit4]   2> 963534 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 963536 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 963536 INFO  (searcherExecutor-4285-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4732f0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604552}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 963537 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1fac9b9[collection1] main]
   [junit4]   2> 963538 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@777ae4
   [junit4]   2> 963538 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=777ae4
   [junit4]   2> 963538 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@f1fc77: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@196998f
   [junit4]   2> 963539 INFO  (searcherExecutor-4285-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1fac9b9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.8.0):c1:[diagnostics={java.runtime.version=1.8.0_172-b11, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os=Windows 10, os.arch=x86, os.version=10.0, source=flush, timestamp=1551753604552}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 963545 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[(null)]} 0 0
   [junit4]   2> 963545 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
   [junit4]   2> 963553 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@10c3bb9[collection1] main]
   [junit4]   2> 963553 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 963554 INFO  (searcherExecutor-4285-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@10c3bb9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.8.0):c1:[diagnostics={java.runtime.version=1.8.0_172-b11, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os=Windows 10, os.arch=x86, os.version=10.0, source=flush, timestamp=1551753604552}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_5(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604836}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 963554 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 9
   [junit4]   2> 963556 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[(null)]} 0 1
   [junit4]   2> 963556 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 963556 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@16a7ff3 commitCommandVersion:0
   [junit4]   2> 963569 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@f68ad0[collection1] main]
   [junit4]   2> 963569 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 963571 INFO  (searcherExecutor-4285-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@f68ad0[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.8.0):c1:[diagnostics={java.runtime.version=1.8.0_172-b11, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os=Windows 10, os.arch=x86, os.version=10.0, source=flush, timestamp=1551753604552}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_5(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604836}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_6(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604847}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 963571 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 14
   [junit4]   2> 963572 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[(null)]} 0 0
   [junit4]   2> 963573 ERROR (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.c.CorePropertiesLocator Couldn't load core descriptor from C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\core.properties:java.nio.file.NoSuchFileException: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\core.properties
   [junit4]   2> 963573 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.c.SolrResourceLoader [null] Added 2 libs to classloader, from paths: [/C:/Users/jenkins/workspace/Lucene-Solr-7.x-Windows/solr/core/src/test-files/solr/collection1/lib, /C:/Users/jenkins/workspace/Lucene-Solr-7.x-Windows/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 963662 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.8.0
   [junit4]   2> 963685 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=minimal
   [junit4]   2> 963694 WARN  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.s.IndexSchema no uniqueKey specified in schema.
   [junit4]   2> 963694 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field null
   [junit4]   2> 963694 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.c.CoreContainer Reloading SolrCore 'collection1' using configuration from instancedir C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1
   [junit4]   2> 963725 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7bfe1b
   [junit4]   2> 963725 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001\init-core-data-001\]
   [junit4]   2> 963838 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 963838 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 963845 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@9ed277[collection1] main]
   [junit4]   2> 963846 WARN  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.r.ManagedResourceStorage Cannot write to config directory C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf; switching to use InMemory storage instead.
   [junit4]   2> 963846 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 963848 INFO  (searcherExecutor-4289-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@9ed277[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.8.0):c1:[diagnostics={java.runtime.version=1.8.0_172-b11, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os=Windows 10, os.arch=x86, os.version=10.0, source=flush, timestamp=1551753604552}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_5(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604836}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_6(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753604847}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_7(7.8.0):c1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os.arch=x86, java.runtime.version=1.8.0_172-b11, source=flush, os.version=10.0, timestamp=1551753605130}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 963856 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 963858 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1fe870b[collection1] main]
   [junit4]   2> 963858 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1c72f65
   [junit4]   2> 963858 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1c72f65
   [junit4]   2> 963858 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4ded58: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@196998f
   [junit4]   2> 963864 INFO  (searcherExecutor-4289-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1fe870b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_4(7.8.0):c1:[diagnostics={java.runtime.version=1.8.0_172-b11, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os=Windows 10, os.arch=x86, os.version=10.0, source=flush, timestamp=1551753604552}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_5(7.8.0):c1:[diagnostics={java.runtime.version=1.8.0_172-b11, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os=Windows 10, os.arch=x86, os.version=10.0, source=flush, timestamp=1551753604836}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_6(7.8.0):c1:[diagnostics={java.runtime.version=1.8.0_172-b11, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os=Windows 10, os.arch=x86, os.version=10.0, source=flush, timestamp=1551753604847}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_7(7.8.0):c1:[diagnostics={java.runtime.version=1.8.0_172-b11, java.vendor=Oracle Corporation, java.version=1.8.0_172, java.vm.version=25.172-b11, lucene.version=7.8.0, os=Windows 10, os.arch=x86, os.version=10.0, source=flush, timestamp=1551753605130}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 963864 INFO  (TEST-TestNRTOpen.testReaderIsNRT-seed#[83ECD4C20377A6F5]) [    x:collection1] o.a.s.SolrTestCaseJ4 ###Ending testReaderIsNRT
   [junit4]   2> 963864 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 963864 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    x:collection1] o.a.s.c.CoreContainer Shutting down CoreContainer instance=24939889
   [junit4]   2> 963864 INFO  (coreCloseExecutor-4293-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@1d62410
   [junit4]   2> 963864 INFO  (coreCloseExecutor-4293-thread-1-processing-x:collection1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1d62410
   [junit4]   2> 963864 INFO  (coreCloseExecutor-4293-thread-1-processing-x:collection1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@8d2bb2: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@196998f
   [junit4]   2> 963874 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 963874 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@81fd6b: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1ebf4c3
   [junit4]   2> 963877 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 963877 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1f296dd: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@16815aa
   [junit4]   2> 963879 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 963879 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5f8b6c: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1141728
   [junit4]   2> 963886 INFO  (SUITE-TestNRTOpen-seed#[83ECD4C20377A6F5]-worker) [    x:collection1] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=false): {}, locale=sv-SE, timezone=Pacific/Enderbury
   [junit4]   2> NOTE: Windows 10 10.0 x86/Oracle Corporation 1.8.0_172 (32-bit)/cpus=3,threads=1,free=223310912,total=356745216
   [junit4]   2> NOTE: All tests run in this JVM: [BlockJoinFacetDistribTest, HttpPartitionTest, RuleEngineTest, CustomHighlightComponentTest, ShardRoutingTest, TestSchemalessBufferedUpdates, TestLegacyFieldCache, SplitShardTest, TestLeaderInitiatedRecoveryThread, TestMiniSolrCloudClusterSSL, TestXmlQParser, TestReplicaProperties, TestCrossCoreJoin, TestSolr4Spatial, TestSerializedLuceneMatchVersion, HdfsSyncSliceTest, ConvertedLegacyTest, TestExceedMaxTermLength, SuggestComponentTest, DocExpirationUpdateProcessorFactoryTest, StatsComponentTest, DocValuesMultiTest, TestSolrCoreProperties, TestDocSet, RandomizedTaggerTest, TestSQLHandler, TestDynamicLoading, ZookeeperStatusHandlerTest, SpatialHeatmapFacetsTest, CSVRequestHandlerTest, PreAnalyzedFieldManagedSchemaCloudTest, AssignTest, TestCloudDeleteByQuery, TestClusterStateMutator, LukeRequestHandlerTest, UpdateRequestProcessorFactoryTest, TestComponentsName, TestCloudJSONFacetSKG, SolrMetricReporterTest, TestCSVLoader, TestSegmentSorting, HighlighterConfigTest, RegexBytesRefFilterTest, TestSimpleQParserPlugin, NoCacheHeaderTest, TlogReplayBufferedWhileIndexingTest, ChaosMonkeySafeLeaderTest, TestRetrieveFieldsOptimizer, TriggerEventQueueTest, TestSolrCoreParser, TestGroupingSearch, ClassificationUpdateProcessorFactoryTest, HdfsDirectoryTest, AtomicUpdateProcessorFactoryTest, MoreLikeThisHandlerTest, AutoAddReplicasIntegrationTest, TestNonDefinedSimilarityFactory, TestStressInPlaceUpdates, TestPhraseSuggestions, NumberUtilsTest, TestDynamicFieldCollectionResource, TestGraphMLResponseWriter, SegmentsInfoRequestHandlerTest, IndexBasedSpellCheckerTest, TestBulkSchemaAPI, TestConfigsApi, MetricsHandlerTest, TestIndexSearcher, OverseerCollectionConfigSetProcessorTest, HdfsTlogReplayBufferedWhileIndexingTest, TestRandomFaceting, TestLazyCores, BigEndianAscendingWordSerializerTest, DeleteLastCustomShardedReplicaTest, TestMissingGroups, TestSimDistributedQueue, TestUnInvertedFieldException, ZkFailoverTest, SortSpecParsingTest, TestQuerySenderNoQuery, TestCustomStream, TestBinaryResponseWriter, TestSubQueryTransformerDistrib, TestJoin, TestManagedSchemaThreadSafety, TestNRTOpen]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestNRTOpen -Dtests.seed=83ECD4C20377A6F5 -Dtests.slow=true -Dtests.locale=sv-SE -Dtests.timezone=Pacific/Enderbury -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.00s J0 | TestNRTOpen (suite) <<<
   [junit4]    > Throwable #1: java.io.IOException: Could not remove the following files (in the order of attempts):
   [junit4]    >    C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001\init-core-data-001: java.nio.file.AccessDeniedException: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001\init-core-data-001
   [junit4]    >    C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.core.TestNRTOpen_83ECD4C20377A6F5-001
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([83ECD4C20377A6F5]:0)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:318)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4] Completed [199/838 (1!)] on J0 in 3.02s, 2 tests, 1 error <<< FAILURES!

[...truncated 47406 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/1014/consoleText

[repro] Revision: c4807a64b156c5946aae8ebd6d8c8e80e428a266

[repro] Ant options: "-Dargs=-server -XX:+UseConcMarkSweepGC"

[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr\core
[repro]       TestNRTOpen
[repro] ant compile-test

[...truncated 3605 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestNRTOpen" -Dtests.showOutput=onerror "-Dargs=-server -XX:+UseConcMarkSweepGC" -Dtests.seed=83ECD4C20377A6F5 -Dtests.slow=true -Dtests.locale=sv-SE -Dtests.timezone=Pacific/Enderbury -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[...truncated 73 lines...]
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.core.TestNRTOpen
[repro] Exiting with code 0

[...truncated 78 lines...]

[JENKINS-EA] Lucene-Solr-7.x-Windows (64bit/jdk-13-ea+8) - Build # 1017 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/1017/
Java: 64bit/jdk-13-ea+8 -XX:+UseCompressedOops -XX:+UseParallelGC

1 tests failed.
FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale

Error Message:
Error from server at http://127.0.0.1:58153/solr/stale_state_test_col: ClusterState says we are the leader (http://127.0.0.1:58153/solr/stale_state_test_col_shard1_replica_n3), but locally we don't think so. Request came from null

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58153/solr/stale_state_test_col: ClusterState says we are the leader (http://127.0.0.1:58153/solr/stale_state_test_col_shard1_replica_n3), but locally we don't think so. Request came from null
	at __randomizedtesting.SeedInfo.seed([F0F100FD52C60ABD:44C09815B12F7C91]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:484)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:414)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1110)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:997)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:237)
	at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale(CloudSolrClientTest.java:835)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:239)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:835)




Build Log:
[...truncated 16206 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrClientTest
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-solrj\test\J1\temp\solr.client.solrj.impl.CloudSolrClientTest_F0F100FD52C60ABD-001\init-core-data-001
   [junit4]   1> 22:04:24.321 [jetty-launcher-9-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:24.321 [jetty-launcher-9-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:24.321 [jetty-launcher-9-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:29.553 [jetty-launcher-80-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:29.560 [jetty-launcher-80-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:29.564 [jetty-launcher-80-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:37.191 [jetty-launcher-151-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:37.195 [jetty-launcher-151-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:37.199 [jetty-launcher-151-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:43.174 [qtp1303929914-482] ERROR org.apache.solr.handler.RequestHandlerBase - org.apache.solr.common.SolrException: Unknown action: foobar
   [junit4]   1> 	at org.apache.solr.handler.admin.ConfigSetsHandler.handleRequestBody(ConfigSetsHandler.java:109)
   [junit4]   1> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
   [junit4]   1> 	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
   [junit4]   1> 	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
   [junit4]   1> 	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)
   [junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:395)
   [junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
   [junit4]   1> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:158)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
   [junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:753)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   1> 	at org.eclipse.jetty.server.Server.handle(Server.java:502)
   [junit4]   1> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
   [junit4]   1> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
   [junit4]   1> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
   [junit4]   1> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
   [junit4]   1> 	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
   [junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
   [junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
   [junit4]   1> 	at java.base/java.lang.Thread.run(Thread.java:835)
   [junit4]   1> 
   [junit4]   1> 22:04:43.231 [qtp1303929914-474] ERROR org.apache.solr.handler.RequestHandlerBase - org.apache.solr.common.SolrException: Unknown action: foobar
   [junit4]   1> 	at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:242)
   [junit4]   1> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
   [junit4]   1> 	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
   [junit4]   1> 	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
   [junit4]   1> 	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)
   [junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:395)
   [junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
   [junit4]   1> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:158)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
   [junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:753)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   1> 	at org.eclipse.jetty.server.Server.handle(Server.java:502)
   [junit4]   1> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
   [junit4]   1> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
   [junit4]   1> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
   [junit4]   1> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
   [junit4]   1> 	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
   [junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
   [junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
   [junit4]   1> 	at java.base/java.lang.Thread.run(Thread.java:835)
   [junit4]   1> 
   [junit4]   1> 22:04:43.343 [qtp1303929914-473] ERROR org.apache.solr.handler.RequestHandlerBase - org.apache.solr.common.SolrException: Unsupported operation: foobar
   [junit4]   1> 	at org.apache.solr.handler.admin.CoreAdminHandler.handleCustomAction(CoreAdminHandler.java:220)
   [junit4]   1> 	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:169)
   [junit4]   1> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
   [junit4]   1> 	at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:735)
   [junit4]   1> 	at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:716)
   [junit4]   1> 	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:496)
   [junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:395)
   [junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
   [junit4]   1> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:158)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
   [junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:753)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   1> 	at org.eclipse.jetty.server.Server.handle(Server.java:502)
   [junit4]   1> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
   [junit4]   1> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
   [junit4]   1> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
   [junit4]   1> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
   [junit4]   1> 	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
   [junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
   [junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
   [junit4]   1> 	at java.base/java.lang.Thread.run(Thread.java:835)
   [junit4]   1> 
   [junit4]   1> 22:04:43.371 [SyncThread:0] ERROR org.apache.zookeeper.server.NIOServerCnxn - Unexpected Exception: 
   [junit4]   1> java.nio.channels.CancelledKeyException: null
   [junit4]   1> 	at sun.nio.ch.SelectionKeyImpl.ensureValid(SelectionKeyImpl.java:71) ~[?:?]
   [junit4]   1> 	at sun.nio.ch.SelectionKeyImpl.interestOps(SelectionKeyImpl.java:90) ~[?:?]
   [junit4]   1> 	at org.apache.zookeeper.server.NIOServerCnxn.internalSendBuffer(NIOServerCnxn.java:188) ~[zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   1> 	at org.apache.zookeeper.server.NIOServerCnxn.sendBuffer(NIOServerCnxn.java:147) [zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   1> 	at org.apache.zookeeper.server.NIOServerCnxn.sendCloseSession(NIOServerCnxn.java:119) [zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   1> 	at org.apache.zookeeper.server.FinalRequestProcessor.processRequest(FinalRequestProcessor.java:394) [zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   1> 	at org.apache.zookeeper.server.SyncRequestProcessor.flush(SyncRequestProcessor.java:199) [zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   1> 	at org.apache.zookeeper.server.SyncRequestProcessor.run(SyncRequestProcessor.java:131) [zookeeper-3.4.13.jar:3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03]
   [junit4]   1> 22:04:44.648 [jetty-launcher-227-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:44.652 [jetty-launcher-227-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:44.652 [jetty-launcher-227-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:53.036 [jetty-launcher-298-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:53.037 [jetty-launcher-298-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:53.037 [jetty-launcher-298-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:56.312 [jetty-launcher-374-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:56.317 [jetty-launcher-374-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:56.314 [jetty-launcher-374-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:58.882 [jetty-launcher-450-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:58.892 [jetty-launcher-450-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:04:58.900 [jetty-launcher-450-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:down live:true
   [junit4]   1> replica:core_node4 rstate:down live:true
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:recovering live:true
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:active live:true
   [junit4]   1> no one is recoverying
   [junit4]   1> replica:core_node4 rstate:active live:true
   [junit4]   1> no one is recoverying
   [junit4]   1> 22:05:03.995 [TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[F0F100FD52C60ABD]] ERROR org.apache.solr.client.solrj.impl.CloudSolrClient - Request to collection [stale_state_test_col] failed due to (404) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58150/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html>
   [junit4]   1> <head>
   [junit4]   1> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
   [junit4]   1> <title>Error 404 Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</title>
   [junit4]   1> </head>
   [junit4]   1> <body><h2>HTTP ERROR 404</h2>
   [junit4]   1> <p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason:
   [junit4]   1> <pre>    Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.14.v20181114</a><hr/>
   [junit4]   1> 
   [junit4]   1> </body>
   [junit4]   1> </html>
   [junit4]   1> , retry=0 commError=false errorCode=404 
   [junit4]   1> 22:05:06.377 [qtp1617107319-1285] ERROR org.apache.solr.update.processor.DistributedUpdateProcessor - ClusterState says we are the leader, but locally we don't think so
   [junit4]   1> 22:05:06.383 [qtp1617107319-1285] ERROR org.apache.solr.handler.RequestHandlerBase - org.apache.solr.common.SolrException: ClusterState says we are the leader (http://127.0.0.1:58153/solr/stale_state_test_col_shard1_replica_n3), but locally we don't think so. Request came from null
   [junit4]   1> 	at org.apache.solr.update.processor.DistributedUpdateProcessor.doDefensiveChecks(DistributedUpdateProcessor.java:631)
   [junit4]   1> 	at org.apache.solr.update.processor.DistributedUpdateProcessor.setupRequest(DistributedUpdateProcessor.java:346)
   [junit4]   1> 	at org.apache.solr.update.processor.DistributedUpdateProcessor.setupRequest(DistributedUpdateProcessor.java:281)
   [junit4]   1> 	at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:674)
   [junit4]   1> 	at org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:110)
   [junit4]   1> 	at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$StreamingCodec.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:327)
   [junit4]   1> 	at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$StreamingCodec.readIterator(JavaBinUpdateRequestCodec.java:280)
   [junit4]   1> 	at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:333)
   [junit4]   1> 	at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:278)
   [junit4]   1> 	at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$StreamingCodec.readNamedList(JavaBinUpdateRequestCodec.java:235)
   [junit4]   1> 	at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:298)
   [junit4]   1> 	at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:278)
   [junit4]   1> 	at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:191)
   [junit4]   1> 	at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:126)
   [junit4]   1> 	at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:123)
   [junit4]   1> 	at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:70)
   [junit4]   1> 	at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
   [junit4]   1> 	at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)
   [junit4]   1> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
   [junit4]   1> 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2551)
   [junit4]   1> 	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:710)
   [junit4]   1> 	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:516)
   [junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:395)
   [junit4]   1> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:341)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
   [junit4]   1> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:158)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
   [junit4]   1> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
   [junit4]   1> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
   [junit4]   1> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
   [junit4]   1> 	at org.eclipse.jetty.server.Server.handle(Server.java:502)
   [junit4]   1> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
   [junit4]   1> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
   [junit4]   1> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
   [junit4]   1> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
   [junit4]   1> 	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
   [junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
   [junit4]   1> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
   [junit4]   1> 	at java.base/java.lang.Thread.run(Thread.java:835)
   [junit4]   1> 
   [junit4]   1> 22:05:06.391 [TEST-CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale-seed#[F0F100FD52C60ABD]] ERROR org.apache.solr.client.solrj.impl.CloudSolrClient - Request to collection [stale_state_test_col] failed due to (503) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58153/solr/stale_state_test_col: ClusterState says we are the leader (http://127.0.0.1:58153/solr/stale_state_test_col_shard1_replica_n3), but locally we don't think so. Request came from null, retry=1 commError=false errorCode=503 
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=CloudSolrClientTest -Dtests.method=testRetryUpdatesWhenClusterStateIsStale -Dtests.seed=F0F100FD52C60ABD -Dtests.slow=true -Dtests.locale=yue-Hant-HK -Dtests.timezone=Pacific/Samoa -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   8.87s J1 | CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58153/solr/stale_state_test_col: ClusterState says we are the leader (http://127.0.0.1:58153/solr/stale_state_test_col_shard1_replica_n3), but locally we don't think so. Request came from null
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([F0F100FD52C60ABD:44C09815B12F7C91]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:484)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:414)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1110)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:997)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
   [junit4]    > 	at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:237)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale(CloudSolrClientTest.java:835)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > 	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:835)
   [junit4]   1> 22:05:07.749 [jetty-launcher-527-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:07.751 [jetty-launcher-527-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:07.751 [jetty-launcher-527-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:13.176 [jetty-launcher-598-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:13.176 [jetty-launcher-598-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:13.176 [jetty-launcher-598-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:20.555 [jetty-launcher-669-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:20.556 [jetty-launcher-669-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:20.557 [jetty-launcher-669-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:26.299 [jetty-launcher-745-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:26.308 [jetty-launcher-745-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:26.367 [jetty-launcher-745-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:31.590 [qtp61929520-2101] ERROR org.apache.solr.servlet.SolrDispatchFilter - Error processing the request. CoreContainer is either not initialized or shutting down.
   [junit4]   1> 22:05:31.596 [qtp61929520-2121] ERROR org.apache.solr.servlet.SolrDispatchFilter - Error processing the request. CoreContainer is either not initialized or shutting down.
   [junit4]   1> 22:05:31.601 [indexFetcher-568-thread-1] ERROR org.apache.solr.handler.IndexFetcher - Error fetching file, doing one retry...:org.apache.solr.common.SolrException: Unable to download _0.fnm completely. Downloaded 0!=1089
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1700)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1580)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1550)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:1030)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:569)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:346)
   [junit4]   1> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:425)
   [junit4]   1> 	at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1171)
   [junit4]   1> 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   [junit4]   1> 	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
   [junit4]   1> 	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
   [junit4]   1> 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   [junit4]   1> 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   [junit4]   1> 	at java.base/java.lang.Thread.run(Thread.java:835)
   [junit4]   1> 
   [junit4]   1> 22:05:31.606 [indexFetcher-558-thread-1] ERROR org.apache.solr.handler.IndexFetcher - Error fetching file, doing one retry...:org.apache.solr.common.SolrException: Unable to download _0.nvm completely. Downloaded 0!=133
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1700)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1580)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1550)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:1030)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:569)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:346)
   [junit4]   1> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:425)
   [junit4]   1> 	at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1171)
   [junit4]   1> 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   [junit4]   1> 	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
   [junit4]   1> 	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
   [junit4]   1> 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   [junit4]   1> 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   [junit4]   1> 	at java.base/java.lang.Thread.run(Thread.java:835)
   [junit4]   1> 
   [junit4]   1> 22:05:31.610 [qtp61929520-2101] ERROR org.apache.solr.servlet.SolrDispatchFilter - Error processing the request. CoreContainer is either not initialized or shutting down.
   [junit4]   1> 22:05:31.610 [qtp61929520-2107] ERROR org.apache.solr.servlet.SolrDispatchFilter - Error processing the request. CoreContainer is either not initialized or shutting down.
   [junit4]   1> 22:05:31.616 [indexFetcher-558-thread-1] ERROR org.apache.solr.handler.IndexFetcher - Error deleting file: _0.nvm
   [junit4]   1> java.io.FileNotFoundException: _0.nvm
   [junit4]   1> 	at org.apache.lucene.store.RAMDirectory.deleteFile(RAMDirectory.java:178) ~[java/:?]
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607) ~[java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$DirectoryFile.delete(IndexFetcher.java:1784) ~[java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1694) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1580) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1555) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:1030) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:569) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:346) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:425) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1171) [java/:?]
   [junit4]   1> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
   [junit4]   1> 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
   [junit4]   1> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
   [junit4]   1> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
   [junit4]   1> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:835) [?:?]
   [junit4]   1> 22:05:31.616 [indexFetcher-568-thread-1] ERROR org.apache.solr.handler.IndexFetcher - Error deleting file: _0.fnm
   [junit4]   1> java.io.FileNotFoundException: _0.fnm
   [junit4]   1> 	at org.apache.lucene.store.RAMDirectory.deleteFile(RAMDirectory.java:178) ~[java/:?]
   [junit4]   1> 	at org.apache.lucene.store.MockDirectoryWrapper.deleteFile(MockDirectoryWrapper.java:607) ~[java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$DirectoryFile.delete(IndexFetcher.java:1784) ~[java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1694) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1580) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1555) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:1030) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:569) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:346) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:425) [java/:?]
   [junit4]   1> 	at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1171) [java/:?]
   [junit4]   1> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
   [junit4]   1> 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
   [junit4]   1> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
   [junit4]   1> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
   [junit4]   1> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
   [junit4]   1> 	at java.lang.Thread.run(Thread.java:835) [?:?]
   [junit4]   1> 22:05:31.621 [indexFetcher-568-thread-1] ERROR org.apache.solr.handler.ReplicationHandler - Index fetch failed :org.apache.solr.common.SolrException: Index fetch failed : 
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:676)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:346)
   [junit4]   1> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:425)
   [junit4]   1> 	at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1171)
   [junit4]   1> 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   [junit4]   1> 	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
   [junit4]   1> 	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
   [junit4]   1> 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   [junit4]   1> 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   [junit4]   1> 	at java.base/java.lang.Thread.run(Thread.java:835)
   [junit4]   1> Caused by: org.apache.lucene.store.AlreadyClosedException: Already closed
   [junit4]   1> 	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:337)
   [junit4]   1> 	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:359)
   [junit4]   1> 	at org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:257)
   [junit4]   1> 	at org.apache.solr.update.DefaultSolrCoreState.changeWriter(DefaultSolrCoreState.java:220)
   [junit4]   1> 	at org.apache.solr.update.DefaultSolrCoreState.openIndexWriter(DefaultSolrCoreState.java:245)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:629)
   [junit4]   1> 	... 9 more
   [junit4]   1> 
   [junit4]   1> 22:05:31.639 [indexFetcher-558-thread-1] ERROR org.apache.solr.handler.ReplicationHandler - Index fetch failed :org.apache.solr.common.SolrException: Unable to download _0.nvm completely. Downloaded 0!=133
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.cleanup(IndexFetcher.java:1700)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.fetch(IndexFetcher.java:1580)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher$FileFetcher.fetchFile(IndexFetcher.java:1555)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.downloadIndexFiles(IndexFetcher.java:1030)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:569)
   [junit4]   1> 	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:346)
   [junit4]   1> 	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:425)
   [junit4]   1> 	at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1171)
   [junit4]   1> 	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
   [junit4]   1> 	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
   [junit4]   1> 	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
   [junit4]   1> 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
   [junit4]   1> 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
   [junit4]   1> 	at java.base/java.lang.Thread.run(Thread.java:835)
   [junit4]   1> 
   [junit4]   1> 22:05:33.264 [jetty-launcher-816-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:33.264 [jetty-launcher-816-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:33.267 [jetty-launcher-816-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:38.499 [jetty-launcher-887-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:38.500 [jetty-launcher-887-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:38.500 [jetty-launcher-887-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:40.861 [jetty-launcher-959-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:40.863 [jetty-launcher-959-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:40.864 [jetty-launcher-959-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:45.151 [jetty-launcher-1030-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:45.156 [jetty-launcher-1030-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:45.156 [jetty-launcher-1030-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:48.810 [jetty-launcher-1101-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:48.811 [jetty-launcher-1101-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 22:05:48.814 [jetty-launcher-1101-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-solrj\test\J1\temp\solr.client.solrj.impl.CloudSolrClientTest_F0F100FD52C60ABD-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=false): {}, locale=yue-Hant-HK, timezone=Pacific/Samoa
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 13-ea (64-bit)/cpus=3,threads=1,free=111023376,total=351272960
   [junit4]   2> NOTE: All tests run in this JVM: [QueryFacetMapTest, CoalesceEvaluatorTest, FieldAnalysisResponseTest, CloudSolrClientTest]
   [junit4] Completed [22/174 (1!)] on J1 in 90.16s, 16 tests, 1 error <<< FAILURES!

[...truncated 41873 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/1017/consoleText

[repro] Revision: fe6df3e48524992bf9b95bc7a8cc5d857f322b49

[repro] Ant options: "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC"

[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr\solrj
[repro]       CloudSolrClientTest
[repro] ant compile-test

[...truncated 2744 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.CloudSolrClientTest" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC" -Dtests.seed=F0F100FD52C60ABD -Dtests.slow=true -Dtests.locale=yue-Hant-HK -Dtests.timezone=Pacific/Samoa -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1

[...truncated 148 lines...]
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.client.solrj.impl.CloudSolrClientTest
[repro] Exiting with code 0

[...truncated 78 lines...]

[JENKINS] Lucene-Solr-7.x-Windows (64bit/jdk1.8.0_172) - Build # 1016 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/1016/
Java: 64bit/jdk1.8.0_172 -XX:+UseCompressedOops -XX:+UseSerialGC

4 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:52606

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:52606
	at __randomizedtesting.SeedInfo.seed([24C406848362B28F:AC90395E2D9EDF77]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:484)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:414)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1110)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:338)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1073)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1047)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
	... 43 more


FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:52607

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:52607
	at __randomizedtesting.SeedInfo.seed([24C406848362B28F:AC90395E2D9EDF77]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:484)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:414)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1110)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:338)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1073)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1047)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
	... 43 more


FAILED:  org.apache.solr.cloud.HttpPartitionTest.test

Error Message:
max version bucket seed not updated after recovery!

Stack Trace:
java.lang.AssertionError: max version bucket seed not updated after recovery!
	at __randomizedtesting.SeedInfo.seed([24C406848362B28F:AC90395E2D9EDF77]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:370)
	at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:148)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1075)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1047)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.client.solrj.impl.CloudSolrClientTest

Error Message:
ObjectTracker found 4 object(s) that were not released!!! [MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)  at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:503)  at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:346)  at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:425)  at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1171)  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:95)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:770)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:967)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1088)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)  at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:359)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:738)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:967)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1088)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)  at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1054)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1088)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)  at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  at java.lang.Thread.run(Thread.java:748)   expected null, but was:<ObjectTracker found 4 object(s) that were not released!!! [MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)  at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:503)  at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:346)  at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:425)  at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1171)  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)  at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:95)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:770)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:967)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1088)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)  at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)  at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:359)  at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:738)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:967)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1088)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)  at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  at java.lang.Thread.run(Thread.java:748)  org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore  at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1054)  at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)  at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)  at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1088)  at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)  at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)  at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)  at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)  at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)  at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  at java.lang.Thread.run(Thread.java:748)  >

Stack Trace:
java.lang.AssertionError: ObjectTracker found 4 object(s) that were not released!!! [MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:503)
	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:346)
	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:425)
	at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1171)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:95)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:770)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:967)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1088)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
	at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:359)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:738)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:967)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1088)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
	at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1054)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1088)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
	at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

 expected null, but was:<ObjectTracker found 4 object(s) that were not released!!! [MockDirectoryWrapper, MockDirectoryWrapper, MockDirectoryWrapper, SolrCore]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:503)
	at org.apache.solr.handler.IndexFetcher.fetchLatestIndex(IndexFetcher.java:346)
	at org.apache.solr.handler.ReplicationHandler.doFetch(ReplicationHandler.java:425)
	at org.apache.solr.handler.ReplicationHandler.lambda$setupPolling$13(ReplicationHandler.java:1171)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
	at org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:95)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:770)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:967)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1088)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
	at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.lucene.store.MockDirectoryWrapper
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.CachingDirectoryFactory.get(CachingDirectoryFactory.java:348)
	at org.apache.solr.core.SolrCore.getNewIndexDir(SolrCore.java:359)
	at org.apache.solr.core.SolrCore.initIndex(SolrCore.java:738)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:967)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1088)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
	at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.core.SolrCore
	at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1054)
	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:874)
	at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1178)
	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1088)
	at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:92)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:360)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:395)
	at org.apache.solr.handler.admin.CoreAdminHandler.lambda$handleRequestBody$0(CoreAdminHandler.java:188)
	at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

>
	at __randomizedtesting.SeedInfo.seed([F73ACEE61D0106B9]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotNull(Assert.java:755)
	at org.junit.Assert.assertNull(Assert.java:737)
	at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:320)
	at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:901)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14243 lines...]
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\init-core-data-001
   [junit4]   2> 1593914 WARN  (SUITE-HttpPartitionTest-seed#[24C406848362B28F]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=102 numCloses=102
   [junit4]   2> 1593914 INFO  (SUITE-HttpPartitionTest-seed#[24C406848362B28F]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1593919 INFO  (SUITE-HttpPartitionTest-seed#[24C406848362B28F]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 1593919 INFO  (SUITE-HttpPartitionTest-seed#[24C406848362B28F]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1593920 INFO  (SUITE-HttpPartitionTest-seed#[24C406848362B28F]-worker) [    ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1593922 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1593923 INFO  (ZkTestServer Run Thread) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1593923 INFO  (ZkTestServer Run Thread) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1594023 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer start zk server on port:64306
   [junit4]   2> 1594023 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:64306
   [junit4]   2> 1594023 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 64306
   [junit4]   2> 1594029 INFO  (zkConnectionManagerCallback-6526-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1594034 INFO  (zkConnectionManagerCallback-6528-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1594035 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer put C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1594037 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer put C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 1594039 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer put C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1594040 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer put C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1594040 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer put C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1594041 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer put C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1594042 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer put C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1594043 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer put C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1594044 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer put C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1594045 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer put C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1594046 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer put C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1594047 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 1594628 WARN  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 1594628 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 1594628 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 1594628 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_172-b11
   [junit4]   2> 1594628 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1594628 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1594628 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1594629 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@24ab786a{/,null,AVAILABLE}
   [junit4]   2> 1594631 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@15faa65a{HTTP/1.1,[http/1.1]}{127.0.0.1:64317}
   [junit4]   2> 1594631 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.e.j.s.Server Started @1594666ms
   [junit4]   2> 1594631 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\tempDir-001/control/data, hostContext=/, hostPort=64316, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\control-001\cores}
   [junit4]   2> 1594631 ERROR (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1594631 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1594631 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.8.0
   [junit4]   2> 1594631 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1594631 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1594631 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2019-03-06T12:31:50.747Z
   [junit4]   2> 1594635 INFO  (zkConnectionManagerCallback-6530-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1594636 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1594636 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\control-001\solr.xml
   [junit4]   2> 1594645 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1594646 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1594647 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2ff0e92f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1594687 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 1594694 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64306/solr
   [junit4]   2> 1594696 INFO  (zkConnectionManagerCallback-6534-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1594700 INFO  (zkConnectionManagerCallback-6536-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1594772 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [n:127.0.0.1:64316_    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:64316_
   [junit4]   2> 1594773 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [n:127.0.0.1:64316_    ] o.a.s.c.Overseer Overseer (id=72111486662017028-127.0.0.1:64316_-n_0000000000) starting
   [junit4]   2> 1594781 INFO  (zkConnectionManagerCallback-6543-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1594784 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [n:127.0.0.1:64316_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64306/solr ready
   [junit4]   2> 1594784 INFO  (OverseerStateUpdate-72111486662017028-127.0.0.1:64316_-n_0000000000) [n:127.0.0.1:64316_    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:64316_
   [junit4]   2> 1594785 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [n:127.0.0.1:64316_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64316_
   [junit4]   2> 1594787 INFO  (zkCallback-6542-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1594787 INFO  (zkCallback-6535-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1594842 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [n:127.0.0.1:64316_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1594872 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [n:127.0.0.1:64316_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ff0e92f
   [junit4]   2> 1594879 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [n:127.0.0.1:64316_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ff0e92f
   [junit4]   2> 1594879 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [n:127.0.0.1:64316_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ff0e92f
   [junit4]   2> 1594881 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [n:127.0.0.1:64316_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\control-001\cores
   [junit4]   2> 1594913 INFO  (zkConnectionManagerCallback-6549-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1594914 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1594915 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64306/solr ready
   [junit4]   2> 1594916 INFO  (SocketProxy-Acceptor-64316) [    ] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=64335,localport=64316], receiveBufferSize:65536
   [junit4]   2> 1594916 INFO  (SocketProxy-Acceptor-64316) [    ] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=64317,localport=64336], receiveBufferSize=65536
   [junit4]   2> 1594918 INFO  (qtp1397324878-23738) [n:127.0.0.1:64316_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:64316_&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1594920 INFO  (OverseerThreadFactory-9165-thread-1-processing-n:127.0.0.1:64316_) [n:127.0.0.1:64316_    ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 1595025 INFO  (SocketProxy-Acceptor-64316) [    ] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=64337,localport=64316], receiveBufferSize:65536
   [junit4]   2> 1595026 INFO  (SocketProxy-Acceptor-64316) [    ] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=64317,localport=64338], receiveBufferSize=65536
   [junit4]   2> 1595026 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_    x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1595027 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_    x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1596056 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.8.0
   [junit4]   2> 1596068 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
   [junit4]   2> 1596172 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 1596190 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
   [junit4]   2> 1596190 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ff0e92f
   [junit4]   2> 1596191 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\control-001\cores\control_collection_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\control-001\cores\control_collection_shard1_replica_n1\data\]
   [junit4]   2> 1596205 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1ea1ac4b
   [junit4]   2> 1596206 WARN  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1596290 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1596291 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1596293 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1596293 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1596294 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1596294 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@755707b0[control_collection_shard1_replica_n1] main]
   [junit4]   2> 1596296 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1596296 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1596296 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1596296 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1627259417302925312
   [junit4]   2> 1596299 INFO  (searcherExecutor-9170-thread-1-processing-n:127.0.0.1:64316_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@755707b0[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1596301 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 1596301 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/control_collection/leaders/shard1
   [junit4]   2> 1596304 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1596304 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1596304 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:64316/control_collection_shard1_replica_n1/
   [junit4]   2> 1596305 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1596305 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:64316/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 1596305 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1596305 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/72111486662017028-core_node2-n_0000000000
   [junit4]   2> 1596307 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:64316/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 1596409 INFO  (zkCallback-6535-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1596409 INFO  (zkCallback-6535-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1596409 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1596413 INFO  (qtp1397324878-23741) [n:127.0.0.1:64316_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1386
   [junit4]   2> 1596415 INFO  (qtp1397324878-23738) [n:127.0.0.1:64316_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 1596511 INFO  (zkCallback-6535-thread-3) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1596511 INFO  (zkCallback-6535-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1596511 INFO  (zkCallback-6535-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1596512 INFO  (qtp1397324878-23738) [n:127.0.0.1:64316_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:64316_&wt=javabin&version=2} status=0 QTime=1594
   [junit4]   2> 1596522 INFO  (zkConnectionManagerCallback-6554-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1596524 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1596525 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64306/solr ready
   [junit4]   2> 1596525 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1596526 INFO  (SocketProxy-Acceptor-64316) [    ] o.a.s.c.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=64348,localport=64316], receiveBufferSize:65536
   [junit4]   2> 1596526 INFO  (SocketProxy-Acceptor-64316) [    ] o.a.s.c.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=64317,localport=64349], receiveBufferSize=65536
   [junit4]   2> 1596526 INFO  (qtp1397324878-23740) [n:127.0.0.1:64316_    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1596529 INFO  (OverseerThreadFactory-9165-thread-2-processing-n:127.0.0.1:64316_) [n:127.0.0.1:64316_    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 1596529 INFO  (OverseerCollectionConfigSetProcessor-72111486662017028-127.0.0.1:64316_-n_0000000000) [n:127.0.0.1:64316_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1596735 WARN  (OverseerThreadFactory-9165-thread-2-processing-n:127.0.0.1:64316_) [n:127.0.0.1:64316_    ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 1596737 INFO  (qtp1397324878-23740) [n:127.0.0.1:64316_    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 1596738 INFO  (qtp1397324878-23740) [n:127.0.0.1:64316_    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=211
   [junit4]   2> 1596739 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase Creating jetty instances pullReplicaCount=0 numOtherReplicas=3
   [junit4]   2> 1597303 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\shard-1-001 of type NRT
   [junit4]   2> 1597304 WARN  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 1597304 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 1597304 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 1597304 INFO  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_172-b11
   [junit4]   2> 1597304 INFO  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1597304 INFO  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1597304 INFO  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1597304 INFO  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5251d86a{/,null,AVAILABLE}
   [junit4]   2> 1597307 INFO  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@4e3028fc{HTTP/1.1,[http/1.1]}{127.0.0.1:64351}
   [junit4]   2> 1597307 INFO  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.Server Started @1597342ms
   [junit4]   2> 1597307 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\tempDir-001/jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=64350, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\shard-1-001\cores}
   [junit4]   2> 1597307 ERROR (closeThreadPool-6555-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1597307 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1597307 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.8.0
   [junit4]   2> 1597307 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1597307 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1597307 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2019-03-06T12:31:53.423Z
   [junit4]   2> 1597314 INFO  (zkConnectionManagerCallback-6557-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1597327 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1597327 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\shard-1-001\solr.xml
   [junit4]   2> 1597340 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1597340 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1597342 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2ff0e92f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1597904 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\shard-2-001 of type NRT
   [junit4]   2> 1597907 WARN  (closeThreadPool-6555-thread-2) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 1597907 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 1597907 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 1597907 INFO  (closeThreadPool-6555-thread-2) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_172-b11
   [junit4]   2> 1597907 INFO  (closeThreadPool-6555-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1597907 INFO  (closeThreadPool-6555-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1597907 INFO  (closeThreadPool-6555-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1597907 INFO  (closeThreadPool-6555-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@333a6920{/,null,AVAILABLE}
   [junit4]   2> 1597911 INFO  (closeThreadPool-6555-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2d762857{HTTP/1.1,[http/1.1]}{127.0.0.1:64360}
   [junit4]   2> 1597911 INFO  (closeThreadPool-6555-thread-2) [    ] o.e.j.s.Server Started @1597947ms
   [junit4]   2> 1597911 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=64359, coreRootDirectory=C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\shard-2-001\cores}
   [junit4]   2> 1597912 ERROR (closeThreadPool-6555-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1597912 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1597912 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.8.0
   [junit4]   2> 1597912 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1597913 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1597913 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2019-03-06T12:31:54.029Z
   [junit4]   2> 1597917 INFO  (zkConnectionManagerCallback-6559-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1597917 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1597917 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1003103dab70009, likely client has closed socket
   [junit4]   2> 1597917 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\shard-2-001\solr.xml
   [junit4]   2> 1597930 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1597930 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1597930 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@2ff0e92f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1597979 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 1597986 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64306/solr
   [junit4]   2> 1597989 INFO  (zkConnectionManagerCallback-6563-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1597994 INFO  (zkConnectionManagerCallback-6565-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1597998 INFO  (closeThreadPool-6555-thread-1) [n:127.0.0.1:64350_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1598002 INFO  (closeThreadPool-6555-thread-1) [n:127.0.0.1:64350_    ] o.a.s.c.ZkController Publish node=127.0.0.1:64350_ as DOWN
   [junit4]   2> 1598003 INFO  (closeThreadPool-6555-thread-1) [n:127.0.0.1:64350_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1598003 INFO  (closeThreadPool-6555-thread-1) [n:127.0.0.1:64350_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64350_
   [junit4]   2> 1598003 INFO  (zkCallback-6535-thread-4) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1598003 INFO  (zkCallback-6542-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1598003 INFO  (zkCallback-6553-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1598048 INFO  (zkCallback-6564-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1598082 INFO  (zkConnectionManagerCallback-6572-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1598084 INFO  (closeThreadPool-6555-thread-1) [n:127.0.0.1:64350_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1598085 INFO  (closeThreadPool-6555-thread-1) [n:127.0.0.1:64350_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64306/solr ready
   [junit4]   2> 1598085 INFO  (closeThreadPool-6555-thread-1) [n:127.0.0.1:64350_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1598125 INFO  (closeThreadPool-6555-thread-1) [n:127.0.0.1:64350_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ff0e92f
   [junit4]   2> 1598135 INFO  (closeThreadPool-6555-thread-1) [n:127.0.0.1:64350_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ff0e92f
   [junit4]   2> 1598135 INFO  (closeThreadPool-6555-thread-1) [n:127.0.0.1:64350_    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2ff0e92f
   [junit4]   2> 1598138 INFO  (closeThreadPool-6555-thread-1) [n:127.0.0.1:64350_    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\shard-1-001\cores
   [junit4]   2> 1598450 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 1598459 INFO  (closeThreadPool-6555-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:64306/solr
   [junit4]   2> 1598465 INFO  (zkConnectionManagerCallback-6577-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1598467 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1003103dab7000d, likely client has closed socket
   [junit4]   2> 1598475 INFO  (zkConnectionManagerCallback-6579-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1598480 INFO  (closeThreadPool-6555-thread-2) [n:127.0.0.1:64359_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1598485 INFO  (closeThreadPool-6555-thread-2) [n:127.0.0.1:64359_    ] o.a.s.c.ZkController Publish node=127.0.0.1:64359_ as DOWN
   [junit4]   2> 1598486 INFO  (closeThreadPool-6555-thread-2) [n:127.0.0.1:64359_    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1598486 INFO  (closeThreadPool-6555-thread-2) [n:127.0.0.1:64359_    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:64359_
   [junit4]   2> 1598488 INFO  (zkCallback-6564-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1598488 INFO  (zkCallback-6535-thread-4) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1598488 INFO  (zkCallback-6553-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1598488 INFO  (zkCallback-6542-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1598489 INFO  (zkCallback-6571-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1598489 INFO  (zkCallback-6578-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1598531 INFO  (OverseerCollectionConfigSetProcessor-72111486662017028-127.0.0.1:64316_-n_0000000000) [n:127.0.0.1:64316_    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1598561 INFO  (zkConnectionManagerCallback-6586-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1598562 INFO  (closeThreadPool-6555-thread-2) [n:127.0.0.1:64359_    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1598563 INFO  (closeThreadPool-6555-thread-2) [n:127.0.0.1:64359_    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:64306/solr ready
   [junit4]   2> 1598563 INFO  (closeThreadPool-6555-thread-2) [n:127.0.0.1:64359_    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1598586 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001\shard-3-001 of type NRT
   [junit4]   2> 1598587 WARN  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 1598587 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 1598587 INFO  (closeThreadPool-6555-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 1598587 INFO  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 1.8.0_172-b11
   [junit4]   2> 1598589 INFO  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1598589 INFO  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1598589 INFO  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1598589 INFO  (closeThreadPool-6555-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f37167d{/,null,AVAILA

[...truncated too long message...]

 Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	3	/solr/aliases.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	3	/solr/clusterstate.json
   [junit4]   2> 	3	/solr/clusterprops.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	3	/solr/live_nodes
   [junit4]   2> 	3	/solr/collections
   [junit4]   2> 
   [junit4]   2> 59018 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:52592
   [junit4]   2> 59018 INFO  (TEST-HttpPartitionTest.test-seed#[24C406848362B28F]) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 52592
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HttpPartitionTest -Dtests.method=test -Dtests.seed=24C406848362B28F -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=pl-PL -Dtests.timezone=Europe/Dublin -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   41.2s J1 | HttpPartitionTest.test <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:52606
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([24C406848362B28F:AC90395E2D9EDF77]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:484)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:414)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1110)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:338)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1073)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1047)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:171)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:141)
   [junit4]    > 	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
   [junit4]    > 	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
   [junit4]    > 	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
   [junit4]    > 	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
   [junit4]    > 	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
   [junit4]    > 	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
   [junit4]    > 	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
   [junit4]    > 	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
   [junit4]    > 	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
   [junit4]    > 	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
   [junit4]    > 	... 43 more
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.HttpPartitionTest_24C406848362B28F-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1369, maxMBSortInHeap=5.78953434819707, sim=RandomSimilarity(queryNorm=true): {}, locale=pl-PL, timezone=Europe/Dublin
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=3,threads=1,free=49361712,total=97583104
   [junit4]   2> NOTE: All tests run in this JVM: [HttpPartitionTest]
   [junit4] Completed [2/5 (2!)] on J1 in 56.17s, 1 test, 1 error <<< FAILURES!
   [junit4] 
   [junit4] HEARTBEAT J0 PID(21232@serv1-vm1): 2019-03-06T13:50:22, stalled for 71.1s at: HttpPartitionTest.test
   [junit4] HEARTBEAT J1 PID(20924@serv1-vm1): 2019-03-06T13:50:22, stalled for 70.7s at: HttpPartitionTest.test
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4] OK      72.2s J0 | HttpPartitionTest.test
   [junit4] Completed [3/5 (2!)] on J0 in 73.08s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4] OK      71.4s J1 | HttpPartitionTest.test
   [junit4] Completed [4/5 (2!)] on J1 in 72.47s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
   [junit4] OK      48.5s J0 | HttpPartitionTest.test
   [junit4] Completed [5/5 (2!)] on J0 in 49.43s, 1 test
   [junit4] 
   [junit4] 
   [junit4] Tests with failures [seed: 24C406848362B28F]:
   [junit4]   - org.apache.solr.cloud.HttpPartitionTest.test
   [junit4]   - org.apache.solr.cloud.HttpPartitionTest.test
   [junit4] 
   [junit4] 
   [junit4] JVM J0:     1.69 ..   183.25 =   181.56s
   [junit4] JVM J1:     1.66 ..   134.19 =   132.53s
   [junit4] Execution time total: 3 minutes 3 seconds
   [junit4] Tests summary: 5 suites, 5 tests, 2 errors

BUILD FAILED
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\common-build.xml:1572: The following error occurred while executing this line:
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\common-build.xml:1099: There were test failures: 5 suites, 5 tests, 2 errors [seed: 24C406848362B28F]

Total time: 3 minutes 7 seconds

[repro] Setting last failure code to 1

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.client.solrj.impl.CloudSolrClientTest
[repro]   2/5 failed: org.apache.solr.cloud.HttpPartitionTest
[repro] Exiting with code 1

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows>IF EXIST lucene\build move /y lucene\build lucene\build.repro 
        1 dir(s) moved.

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows>IF EXIST solr\build move /y solr\build solr\build.repro 
        1 dir(s) moved.

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows>IF EXIST lucene\build.orig move /y lucene\build.orig lucene\build 
        1 dir(s) moved.

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows>IF EXIST solr\build.orig move /y solr\build.orig solr\build 
        1 dir(s) moved.

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows>exit 0 
Archiving artifacts
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
No credentials specified
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=393afa1370eeabb67f678e29d5736ddc7f0d1a85, workspace=C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows
[WARNINGS] Computing warning deltas based on reference build #1015
Recording test results
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2

[JENKINS] Lucene-Solr-7.x-Windows (64bit/jdk-9.0.4) - Build # 1015 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Windows/1015/
Java: 64bit/jdk-9.0.4 -XX:+UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderTragicEventTest.test

Error Message:
Failed while waiting for active collection Timeout waiting to see state for collection=collection1 :null Live Nodes: [127.0.0.1:55232_solr, 127.0.0.1:55235_solr] Last available state: null

Stack Trace:
java.lang.RuntimeException: Failed while waiting for active collection
Timeout waiting to see state for collection=collection1 :null
Live Nodes: [127.0.0.1:55232_solr, 127.0.0.1:55235_solr]
Last available state: null
	at __randomizedtesting.SeedInfo.seed([179C25DD2B62C249:9FC81A07859EAFB1]:0)
	at org.apache.solr.cloud.MiniSolrCloudCluster.waitForActiveCollection(MiniSolrCloudCluster.java:728)
	at org.apache.solr.cloud.MiniSolrCloudCluster.waitForActiveCollection(MiniSolrCloudCluster.java:734)
	at org.apache.solr.cloud.LeaderTragicEventTest.test(LeaderTragicEventTest.java:79)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.LeaderTragicEventTest.test

Error Message:
Failed while waiting for active collection Timeout waiting to see state for collection=collection1 :null Live Nodes: [127.0.0.1:55498_solr, 127.0.0.1:55499_solr] Last available state: null

Stack Trace:
java.lang.RuntimeException: Failed while waiting for active collection
Timeout waiting to see state for collection=collection1 :null
Live Nodes: [127.0.0.1:55498_solr, 127.0.0.1:55499_solr]
Last available state: null
	at __randomizedtesting.SeedInfo.seed([179C25DD2B62C249:9FC81A07859EAFB1]:0)
	at org.apache.solr.cloud.MiniSolrCloudCluster.waitForActiveCollection(MiniSolrCloudCluster.java:728)
	at org.apache.solr.cloud.MiniSolrCloudCluster.waitForActiveCollection(MiniSolrCloudCluster.java:734)
	at org.apache.solr.cloud.LeaderTragicEventTest.test(LeaderTragicEventTest.java:79)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 13492 lines...]
   [junit4] Suite: org.apache.solr.cloud.LeaderTragicEventTest
   [junit4]   2> 1090781 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderTragicEventTest_179C25DD2B62C249-001\init-core-data-001
   [junit4]   2> 1090783 WARN  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
   [junit4]   2> 1090783 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1090785 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 1090785 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderTragicEventTest_179C25DD2B62C249-001\tempDir-001
   [junit4]   2> 1090787 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1090787 INFO  (ZkTestServer Run Thread) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1090787 INFO  (ZkTestServer Run Thread) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1090888 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:55487
   [junit4]   2> 1090888 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:55487
   [junit4]   2> 1090888 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 55487
   [junit4]   2> 1090895 INFO  (zkConnectionManagerCallback-8004-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1090900 INFO  (zkConnectionManagerCallback-8006-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1090904 INFO  (zkConnectionManagerCallback-8008-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1090907 WARN  (jetty-launcher-8009-thread-1) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 1090907 WARN  (jetty-launcher-8009-thread-2) [    ] o.e.j.s.AbstractConnector Ignoring deprecated socket close linger time
   [junit4]   2> 1090907 INFO  (jetty-launcher-8009-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 1090907 INFO  (jetty-launcher-8009-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Start Jetty (original configured port=0)
   [junit4]   2> 1090907 INFO  (jetty-launcher-8009-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 1090907 INFO  (jetty-launcher-8009-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
   [junit4]   2> 1090907 INFO  (jetty-launcher-8009-thread-2) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 9.0.4+11
   [junit4]   2> 1090907 INFO  (jetty-launcher-8009-thread-1) [    ] o.e.j.s.Server jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 9.0.4+11
   [junit4]   2> 1090908 INFO  (jetty-launcher-8009-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1090908 INFO  (jetty-launcher-8009-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1090908 INFO  (jetty-launcher-8009-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1090908 INFO  (jetty-launcher-8009-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1090908 INFO  (jetty-launcher-8009-thread-1) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1090908 INFO  (jetty-launcher-8009-thread-2) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1090909 INFO  (jetty-launcher-8009-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@83320b1{/solr,null,AVAILABLE}
   [junit4]   2> 1090909 INFO  (jetty-launcher-8009-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b561a3{/solr,null,AVAILABLE}
   [junit4]   2> 1090910 INFO  (jetty-launcher-8009-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7cc30645{HTTP/1.1,[http/1.1]}{127.0.0.1:55498}
   [junit4]   2> 1090910 INFO  (jetty-launcher-8009-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@63714c93{HTTP/1.1,[http/1.1]}{127.0.0.1:55499}
   [junit4]   2> 1090910 INFO  (jetty-launcher-8009-thread-1) [    ] o.e.j.s.Server Started @1090972ms
   [junit4]   2> 1090910 INFO  (jetty-launcher-8009-thread-2) [    ] o.e.j.s.Server Started @1090972ms
   [junit4]   2> 1090910 INFO  (jetty-launcher-8009-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55499}
   [junit4]   2> 1090910 INFO  (jetty-launcher-8009-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55498}
   [junit4]   2> 1090910 ERROR (jetty-launcher-8009-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1090910 ERROR (jetty-launcher-8009-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1090910 INFO  (jetty-launcher-8009-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1090910 INFO  (jetty-launcher-8009-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.8.0
   [junit4]   2> 1090910 INFO  (jetty-launcher-8009-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1090910 INFO  (jetty-launcher-8009-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1090910 INFO  (jetty-launcher-8009-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1090911 INFO  (jetty-launcher-8009-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2019-03-05T17:13:53.376068500Z
   [junit4]   2> 1090911 INFO  (jetty-launcher-8009-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.8.0
   [junit4]   2> 1090911 INFO  (jetty-launcher-8009-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1090911 INFO  (jetty-launcher-8009-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1090911 INFO  (jetty-launcher-8009-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2019-03-05T17:13:53.376068500Z
   [junit4]   2> 1090915 INFO  (zkConnectionManagerCallback-8013-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1090915 INFO  (zkConnectionManagerCallback-8012-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1090916 INFO  (jetty-launcher-8009-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1090916 INFO  (jetty-launcher-8009-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1090920 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1002cdfb8340004, likely client has closed socket
   [junit4]   2> 1090990 INFO  (jetty-launcher-8009-thread-2) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1090994 INFO  (jetty-launcher-8009-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55487/solr
   [junit4]   2> 1090999 INFO  (zkConnectionManagerCallback-8017-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1091000 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1002cdfb8340005, likely client has closed socket
   [junit4]   2> 1091003 INFO  (zkConnectionManagerCallback-8019-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1091120 INFO  (jetty-launcher-8009-thread-2) [n:127.0.0.1:55499_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55499_solr
   [junit4]   2> 1091121 INFO  (jetty-launcher-8009-thread-2) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer Overseer (id=72106933417672710-127.0.0.1:55499_solr-n_0000000000) starting
   [junit4]   2> 1091128 INFO  (zkConnectionManagerCallback-8026-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1091131 INFO  (jetty-launcher-8009-thread-2) [n:127.0.0.1:55499_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55487/solr ready
   [junit4]   2> 1091132 INFO  (jetty-launcher-8009-thread-2) [n:127.0.0.1:55499_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55499_solr
   [junit4]   2> 1091165 INFO  (jetty-launcher-8009-thread-2) [n:127.0.0.1:55499_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1091195 INFO  (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:55499_solr
   [junit4]   2> 1091195 INFO  (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1091208 INFO  (jetty-launcher-8009-thread-2) [n:127.0.0.1:55499_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55499.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@355917f4
   [junit4]   2> 1091217 INFO  (jetty-launcher-8009-thread-2) [n:127.0.0.1:55499_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55499.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@355917f4
   [junit4]   2> 1091217 INFO  (jetty-launcher-8009-thread-2) [n:127.0.0.1:55499_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55499.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@355917f4
   [junit4]   2> 1091219 INFO  (jetty-launcher-8009-thread-2) [n:127.0.0.1:55499_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderTragicEventTest_179C25DD2B62C249-001\tempDir-001\node2\.
   [junit4]   2> 1091312 INFO  (zkCallback-8025-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1091511 INFO  (jetty-launcher-8009-thread-1) [    ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1091514 INFO  (jetty-launcher-8009-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:55487/solr
   [junit4]   2> 1091517 INFO  (zkConnectionManagerCallback-8031-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1091518 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1002cdfb8340008, likely client has closed socket
   [junit4]   2> 1091520 INFO  (zkConnectionManagerCallback-8033-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1091525 INFO  (jetty-launcher-8009-thread-1) [n:127.0.0.1:55498_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1091529 INFO  (jetty-launcher-8009-thread-1) [n:127.0.0.1:55498_solr    ] o.a.s.c.ZkController Publish node=127.0.0.1:55498_solr as DOWN
   [junit4]   2> 1091529 INFO  (jetty-launcher-8009-thread-1) [n:127.0.0.1:55498_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1091529 INFO  (jetty-launcher-8009-thread-1) [n:127.0.0.1:55498_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55498_solr
   [junit4]   2> 1091529 DEBUG (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:55498_solr"} current state version: 0
   [junit4]   2> 1091531 INFO  (zkCallback-8018-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1091531 INFO  (zkCallback-8025-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1091532 INFO  (zkCallback-8032-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1091558 INFO  (zkConnectionManagerCallback-8040-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1091560 INFO  (jetty-launcher-8009-thread-1) [n:127.0.0.1:55498_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1091561 INFO  (jetty-launcher-8009-thread-1) [n:127.0.0.1:55498_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55487/solr ready
   [junit4]   2> 1091561 INFO  (jetty-launcher-8009-thread-1) [n:127.0.0.1:55498_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1091586 INFO  (jetty-launcher-8009-thread-1) [n:127.0.0.1:55498_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55498.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@355917f4
   [junit4]   2> 1091596 INFO  (jetty-launcher-8009-thread-1) [n:127.0.0.1:55498_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55498.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@355917f4
   [junit4]   2> 1091596 INFO  (jetty-launcher-8009-thread-1) [n:127.0.0.1:55498_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55498.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@355917f4
   [junit4]   2> 1091597 INFO  (jetty-launcher-8009-thread-1) [n:127.0.0.1:55498_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderTragicEventTest_179C25DD2B62C249-001\tempDir-001\node1\.
   [junit4]   2> 1091630 INFO  (zkConnectionManagerCallback-8046-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1091631 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1091633 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:55487/solr ready
   [junit4]   2> 1091651 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testOtherReplicasAreNotActive
   [junit4]   2> 1091653 INFO  (qtp1403782476-20861) [n:127.0.0.1:55498_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=config&name=collection2&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1091656 INFO  (OverseerThreadFactory-7294-thread-1-processing-n:127.0.0.1:55499_solr) [n:127.0.0.1:55499_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection2
   [junit4]   2> 1091660 DEBUG (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "name":"collection2",
   [junit4]   2>   "fromApi":"true",
   [junit4]   2>   "collection.configName":"config",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "stateFormat":"2",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "operation":"create"} current state version: 0
   [junit4]   2> 1091766 DEBUG (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection2_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:55499/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 1091767 INFO  (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection2_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:55499/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1091771 DEBUG (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection2_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:55498/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} current state version: 0
   [junit4]   2> 1091771 INFO  (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"collection2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"collection2_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:55498/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1091978 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr    x:collection2_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=collection2_shard1_replica_n1&action=CREATE&numShards=1&collection=collection2&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1091979 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr    x:collection2_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1091984 DEBUG (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection2_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:55499/solr",
   [junit4]   2>   "node_name":"127.0.0.1:55499_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"collection2",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "force_set_state":"false",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1091988 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr    x:collection2_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=collection2_shard1_replica_n2&action=CREATE&numShards=1&collection=collection2&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1091998 DEBUG (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
   [junit4]   2>   "core":"collection2_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node4",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:55498/solr",
   [junit4]   2>   "node_name":"127.0.0.1:55498_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"collection2",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "force_set_state":"false",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1093002 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.8.0
   [junit4]   2> 1093010 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.IndexSchema [collection2_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 1093013 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1093013 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection2_shard1_replica_n1' using configuration from collection collection2, trusted=true
   [junit4]   2> 1093014 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55499.solr.core.collection2.shard1.replica_n1' (registry 'solr.core.collection2.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@355917f4
   [junit4]   2> 1093014 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SolrCore [[collection2_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderTragicEventTest_179C25DD2B62C249-001\tempDir-001\node2\collection2_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderTragicEventTest_179C25DD2B62C249-001\tempDir-001\node2\.\collection2_shard1_replica_n1\data\]
   [junit4]   2> 1093033 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.8.0
   [junit4]   2> 1093040 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.s.IndexSchema [collection2_shard1_replica_n2] Schema name=minimal
   [junit4]   2> 1093042 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1093042 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection2_shard1_replica_n2' using configuration from collection collection2, trusted=true
   [junit4]   2> 1093043 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55498.solr.core.collection2.shard1.replica_n2' (registry 'solr.core.collection2.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@355917f4
   [junit4]   2> 1093043 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.c.SolrCore [[collection2_shard1_replica_n2] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderTragicEventTest_179C25DD2B62C249-001\tempDir-001\node1\collection2_shard1_replica_n2], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderTragicEventTest_179C25DD2B62C249-001\tempDir-001\node1\.\collection2_shard1_replica_n2\data\]
   [junit4]   2> 1093149 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1093149 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1093152 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1093152 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1093155 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@51062394[collection2_shard1_replica_n1] main]
   [junit4]   2> 1093156 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 1093156 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 1093159 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1093159 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1627186565610471424
   [junit4]   2> 1093161 INFO  (searcherExecutor-7302-thread-1-processing-n:127.0.0.1:55499_solr x:collection2_shard1_replica_n1 c:collection2 s:shard1 r:core_node3) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SolrCore [collection2_shard1_replica_n1] Registered new searcher Searcher@51062394[collection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1093166 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection2/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 1093166 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection2/leaders/shard1
   [junit4]   2> 1093168 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1093168 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1093171 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 1093172 DEBUG (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"collection2"} current state version: 0
   [junit4]   2> 1093173 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1093173 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1093175 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@472e6770[collection2_shard1_replica_n2] main]
   [junit4]   2> 1093177 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
   [junit4]   2> 1093177 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
   [junit4]   2> 1093178 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1093178 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1627186565630394368
   [junit4]   2> 1093181 INFO  (searcherExecutor-7303-thread-1-processing-n:127.0.0.1:55498_solr x:collection2_shard1_replica_n2 c:collection2 s:shard1 r:core_node4) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.c.SolrCore [collection2_shard1_replica_n2] Registered new searcher Searcher@472e6770[collection2_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1093184 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection2/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
   [junit4]   2> 1093184 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection2/leaders/shard1
   [junit4]   2> 1093673 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1093673 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1093673 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:55499/solr/collection2_shard1_replica_n1/
   [junit4]   2> 1093677 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=collection2_shard1_replica_n1 url=http://127.0.0.1:55499/solr START replicas=[http://127.0.0.1:55498/solr/collection2_shard1_replica_n2/] nUpdates=100
   [junit4]   2> 1093677 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=collection2_shard1_replica_n1 url=http://127.0.0.1:55499/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 1093681 INFO  (qtp1403782476-20875) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.c.S.Request [collection2_shard1_replica_n2]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 1093683 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 1093683 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 1093683 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1093683 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection2/leaders/shard1/leader after winning as /collections/collection2/leader_elect/shard1/election/72106933417672710-core_node3-n_0000000000
   [junit4]   2> 1093687 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:55499/solr/collection2_shard1_replica_n1/ shard1
   [junit4]   2> 1093690 DEBUG (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"leader",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"collection2",
   [junit4]   2>   "base_url":"http://127.0.0.1:55499/solr",
   [junit4]   2>   "core":"collection2_shard1_replica_n1",
   [junit4]   2>   "state":"active"} current state version: 0
   [junit4]   2> 1093794 INFO  (zkCallback-8018-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1093795 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1093804 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=collection2_shard1_replica_n1&action=CREATE&numShards=1&collection=collection2&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1827
   [junit4]   2> 1093805 DEBUG (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection2_shard1_replica_n1",
   [junit4]   2>   "core_node_name":"core_node3",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:55499/solr",
   [junit4]   2>   "node_name":"127.0.0.1:55499_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"collection2",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "force_set_state":"false",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1093907 INFO  (zkCallback-8018-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1093907 INFO  (zkCallback-8018-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1094191 DEBUG (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "core":"collection2_shard1_replica_n2",
   [junit4]   2>   "core_node_name":"core_node4",
   [junit4]   2>   "roles":null,
   [junit4]   2>   "base_url":"http://127.0.0.1:55498/solr",
   [junit4]   2>   "node_name":"127.0.0.1:55498_solr",
   [junit4]   2>   "numShards":"1",
   [junit4]   2>   "state":"active",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "collection":"collection2",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "force_set_state":"false",
   [junit4]   2>   "operation":"state"} current state version: 0
   [junit4]   2> 1094194 INFO  (qtp1403782476-20867) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=collection2_shard1_replica_n2&action=CREATE&numShards=1&collection=collection2&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2206
   [junit4]   2> 1094199 INFO  (qtp1403782476-20861) [n:127.0.0.1:55498_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 1094295 INFO  (zkCallback-8018-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1094295 INFO  (zkCallback-8018-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1094296 INFO  (zkCallback-8032-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1094296 INFO  (qtp1403782476-20861) [n:127.0.0.1:55498_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=config&name=collection2&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2643
   [junit4]   2> 1094299 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.c.LeaderTragicEventTest Stop jetty node : http://127.0.0.1:55498/solr state:DocCollection(collection2//collections/collection2/state.json/6)={
   [junit4]   2>   "pullReplicas":"0",
   [junit4]   2>   "replicationFactor":"2",
   [junit4]   2>   "shards":{"shard1":{
   [junit4]   2>       "range":"80000000-7fffffff",
   [junit4]   2>       "state":"active",
   [junit4]   2>       "replicas":{
   [junit4]   2>         "core_node3":{
   [junit4]   2>           "core":"collection2_shard1_replica_n1",
   [junit4]   2>           "base_url":"http://127.0.0.1:55499/solr",
   [junit4]   2>           "node_name":"127.0.0.1:55499_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "force_set_state":"false",
   [junit4]   2>           "leader":"true"},
   [junit4]   2>         "core_node4":{
   [junit4]   2>           "core":"collection2_shard1_replica_n2",
   [junit4]   2>           "base_url":"http://127.0.0.1:55498/solr",
   [junit4]   2>           "node_name":"127.0.0.1:55498_solr",
   [junit4]   2>           "state":"active",
   [junit4]   2>           "type":"NRT",
   [junit4]   2>           "force_set_state":"false"}}}},
   [junit4]   2>   "router":{"name":"compositeId"},
   [junit4]   2>   "maxShardsPerNode":"1",
   [junit4]   2>   "autoAddReplicas":"false",
   [junit4]   2>   "nrtReplicas":"2",
   [junit4]   2>   "tlogReplicas":"0"}
   [junit4]   2> 1094301 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1839854035
   [junit4]   2> 1094301 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:55498_solr
   [junit4]   2> 1094301 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.c.CoreContainer Publish this node as DOWN...
   [junit4]   2> 1094301 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.c.ZkController Publish node=127.0.0.1:55498_solr as DOWN
   [junit4]   2> 1094306 INFO  (zkCallback-8032-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1094309 DEBUG (OverseerStateUpdate-72106933417672710-127.0.0.1:55499_solr-n_0000000000) [n:127.0.0.1:55499_solr    ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
   [junit4]   2>   "operation":"downnode",
   [junit4]   2>   "node_name":"127.0.0.1:55498_solr"} current state version: 0
   [junit4]   2> 1094309 INFO  (coreCloseExecutor-7312-thread-1) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.c.SolrCore [collection2_shard1_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@1b063ef1
   [junit4]   2> 1094309 INFO  (coreCloseExecutor-7312-thread-1) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection2.shard1.replica_n2, tag=1b063ef1
   [junit4]   2> 1094309 INFO  (coreCloseExecutor-7312-thread-1) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5938a78f: rootName = solr_55498, domain = solr.core.collection2.shard1.replica_n2, service url = null, agent id = null] for registry solr.core.collection2.shard1.replica_n2 / com.codahale.metrics.MetricRegistry@63b91d88
   [junit4]   2> 1094319 INFO  (coreCloseExecutor-7312-thread-1) [n:127.0.0.1:55498_solr c:collection2 s:shard1 r:core_node4 x:collection2_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection2.shard1.leader, tag=1b063ef1
   [junit4]   2> 1094321 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1094321 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@696a9d33: rootName = solr_55498, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@10455172
   [junit4]   2> 1094330 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1094330 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2c47101b: rootName = solr_55498, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@378833a6
   [junit4]   2> 1094336 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1094336 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@74734582: rootName = solr_55498, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@65ab6359
   [junit4]   2> 1094336 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1094339 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1002cdfb8340009, likely client has closed socket
   [junit4]   2> 1094341 INFO  (zkCallback-8018-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1094341 INFO  (zkCallback-8039-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1094341 INFO  (zkCallback-8025-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1094341 INFO  (zkCallback-8045-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1094355 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@7cc30645{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1094355 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2b561a3{/solr,null,UNAVAILABLE}
   [junit4]   2> 1094355 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1094410 INFO  (zkCallback-8018-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1094410 INFO  (zkCallback-8018-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1094410 INFO  (zkCallback-8045-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2/state.json] for collection [collection2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1094413 INFO  (TEST-LeaderTragicEventTest.testOtherReplicasAreNotActive-seed#[179C25DD2B62C249]) [    ] o.a.s.c.LeaderTragicEventTest Corrupt leader : core_node3:{"core":"collection2_shard1_replica_n1","base_url":"http://127.0.0.1:55499/solr","node_name":"127.0.0.1:55499_solr","state":"active","type":"NRT","force_set_state":"false","leader":"true"}
   [junit4]   2> 1094426 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection2/terms/shard1 to Terms{values={core_node3=1, core_node4=0}, version=2}
   [junit4]   2> 1094426 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0 (1627186566931677184)]} 0 8
   [junit4]   2> 1094565 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1627186567084769280,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1094565 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@422b6769 commitCommandVersion:1627186567084769280
   [junit4]   2> 1094619 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3951059b[collection2_shard1_replica_n1] main]
   [junit4]   2> 1094620 INFO  (searcherExecutor-7302-thread-1-processing-n:127.0.0.1:55499_solr x:collection2_shard1_replica_n1 c:collection2 s:shard1 r:core_node3) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SolrCore [collection2_shard1_replica_n1] Registered new searcher Searcher@3951059b[collection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037082}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 1094621 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1094621 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 193
   [junit4]   2> 1094623 INFO  (qtp1159956035-20868) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1 (1627186567143489536)]} 0 2
   [junit4]   2> 1094625 INFO  (qtp1159956035-20868) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1627186567147683840,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1094625 INFO  (qtp1159956035-20868) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@422b6769 commitCommandVersion:1627186567147683840
   [junit4]   2> 1094626 INFO  (qtp1159956035-20868) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2d8a632a[collection2_shard1_replica_n1] main]
   [junit4]   2> 1094627 INFO  (searcherExecutor-7302-thread-1-processing-n:127.0.0.1:55499_solr x:collection2_shard1_replica_n1 c:collection2 s:shard1 r:core_node3) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SolrCore [collection2_shard1_replica_n1] Registered new searcher Searcher@2d8a632a[collection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037082}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_1(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037090}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 1094629 INFO  (qtp1159956035-20868) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1094629 INFO  (qtp1159956035-20868) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 4
   [junit4]   2> 1094633 INFO  (qtp1159956035-20870) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[2 (1627186567153975296)]} 0 1
   [junit4]   2> 1094633 INFO  (qtp1159956035-20870) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1627186567156072448,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1094633 INFO  (qtp1159956035-20870) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@422b6769 commitCommandVersion:1627186567156072448
   [junit4]   2> 1094640 INFO  (qtp1159956035-20870) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5b02597c[collection2_shard1_replica_n1] main]
   [junit4]   2> 1094642 INFO  (searcherExecutor-7302-thread-1-processing-n:127.0.0.1:55499_solr x:collection2_shard1_replica_n1 c:collection2 s:shard1 r:core_node3) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SolrCore [collection2_shard1_replica_n1] Registered new searcher Searcher@5b02597c[collection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037082}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_1(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037090}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_2(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037100}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 1094642 INFO  (qtp1159956035-20870) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1094643 INFO  (qtp1159956035-20870) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 9
   [junit4]   2> 1094645 INFO  (qtp1159956035-20874) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[3 (1627186567167606784)]} 0 1
   [junit4]   2> 1094646 INFO  (qtp1159956035-20874) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1627186567169703936,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1094646 INFO  (qtp1159956035-20874) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@422b6769 commitCommandVersion:1627186567169703936
   [junit4]   2> 1094646 INFO  (qtp1159956035-20874) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2b313a2[collection2_shard1_replica_n1] main]
   [junit4]   2> 1094650 INFO  (searcherExecutor-7302-thread-1-processing-n:127.0.0.1:55499_solr x:collection2_shard1_replica_n1 c:collection2 s:shard1 r:core_node3) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SolrCore [collection2_shard1_replica_n1] Registered new searcher Searcher@2b313a2[collection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037082}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_1(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037090}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_2(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037100}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_3(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037111}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 1094650 INFO  (qtp1159956035-20874) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1094650 INFO  (qtp1159956035-20874) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 4
   [junit4]   2> 1094652 INFO  (qtp1159956035-20876) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[4 (1627186567173898240)]} 0 1
   [junit4]   2> 1094652 INFO  (qtp1159956035-20876) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1627186567175995392,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1094653 INFO  (qtp1159956035-20876) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@422b6769 commitCommandVersion:1627186567175995392
   [junit4]   2> 1094655 INFO  (qtp1159956035-20876) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@741c4ad[collection2_shard1_replica_n1] main]
   [junit4]   2> 1094657 INFO  (qtp1159956035-20876) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1094657 INFO  (searcherExecutor-7302-thread-1-processing-n:127.0.0.1:55499_solr x:collection2_shard1_replica_n1 c:collection2 s:shard1 r:core_node3) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SolrCore [collection2_shard1_replica_n1] Registered new searcher Searcher@741c4ad[collection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037082}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_1(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037090}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_2(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037100}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_3(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037111}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_4(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037118}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 1094657 INFO  (qtp1159956035-20876) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 4
   [junit4]   2> 1094661 INFO  (qtp1159956035-20860) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[5 (1627186567184384000)]} 0 1
   [junit4]   2> 1094661 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1627186567185432576,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1094661 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@422b6769 commitCommandVersion:1627186567185432576
   [junit4]   2> 1094664 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1561df9f[collection2_shard1_replica_n1] main]
   [junit4]   2> 1094665 INFO  (searcherExecutor-7302-thread-1-processing-n:127.0.0.1:55499_solr x:collection2_shard1_replica_n1 c:collection2 s:shard1 r:core_node3) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SolrCore [collection2_shard1_replica_n1] Registered new searcher Searcher@1561df9f[collection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037082}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_1(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037090}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_2(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037100}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_3(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037111}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_4(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037118}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_5(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037127}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 1094665 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1094666 INFO  (qtp1159956035-20862) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 3
   [junit4]   2> 1094667 INFO  (qtp1159956035-20866) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[6 (1627186567190675456)]} 0 1
   [junit4]   2> 1094667 INFO  (qtp1159956035-20866) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1627186567191724032,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1094669 INFO  (qtp1159956035-20866) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@422b6769 commitCommandVersion:1627186567191724032
   [junit4]   2> 1094673 INFO  (qtp1159956035-20866) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1cb24b6c[collection2_shard1_replica_n1] main]
   [junit4]   2> 1094674 INFO  (searcherExecutor-7302-thread-1-processing-n:127.0.0.1:55499_solr x:collection2_shard1_replica_n1 c:collection2 s:shard1 r:core_node3) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SolrCore [collection2_shard1_replica_n1] Registered new searcher Searcher@1cb24b6c[collection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037082}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_1(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037090}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_2(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037100}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_3(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037111}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_4(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037118}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_5(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037127}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_6(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037134}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 1094675 INFO  (qtp1159956035-20866) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1094675 INFO  (qtp1159956035-20866) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 6
   [junit4]   2> 1094678 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[7 (1627186567201161216)]} 0 1
   [junit4]   2> 1094679 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1627186567204306944,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1094679 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@422b6769 commitCommandVersion:1627186567204306944
   [junit4]   2> 1094681 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3caafe4b[collection2_shard1_replica_n1] main]
   [junit4]   2> 1094683 INFO  (searcherExecutor-7302-thread-1-processing-n:127.0.0.1:55499_solr x:collection2_shard1_replica_n1 c:collection2 s:shard1 r:core_node3) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SolrCore [collection2_shard1_replica_n1] Registered new searcher Searcher@3caafe4b[collection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037082}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_1(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037090}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_2(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037100}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_3(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037111}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_4(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037118}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_5(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037127}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_6(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037134}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}]) Uninverting(_7(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.version=9.0.4+11, lucene.version=7.8.0, os.arch=amd64, java.runtime.version=9.0.4+11, source=flush, os.version=10.0, timestamp=1551806037144}]:[attributes={Lucene50StoredFieldsFormat.mode=BEST_SPEED}])))}
   [junit4]   2> 1094684 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1094684 INFO  (qtp1159956035-20864) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 5
   [junit4]   2> 1094688 INFO  (qtp1159956035-20868) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection2_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[8 (1627186567211646976)]} 0 1
   [junit4]   2> 1094688 INFO  (qtp1159956035-20868) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1627186567213744128,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1094689 INFO  (qtp1159956035-20868) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@422b6769 commitCommandVersion:1627186567213744128
   [junit4]   2> 1094692 INFO  (qtp1159956035-20868) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4ffd40d4[collection2_shard1_replica_n1] main]
   [junit4]   2> 1094693 INFO  (searcherExecutor-7302-thread-1-processing-n:127.0.0.1:55499_solr x:collection2_shard1_replica_n1 c:collection2 s:shard1 r:core_node3) [n:127.0.0.1:55499_solr c:collection2 s:shard1 r:core_node3 x:collection2_shard1_replica_n1] o.a.s.c.SolrCore [collection2_shard1_replica_n1] Registered new searcher Searcher@4ffd40d4[collection2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.8.0):C1:[diagnostics={os=Windows 10, java.vendor=Oracle Corporation, java.version=9.0.4, java.vm.vers

[...truncated too long message...]

700 INFO  (jetty-closer-213-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 135700 INFO  (jetty-closer-213-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@531dd8b5: rootName = solr_55235, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4efd1bc3
   [junit4]   2> 135702 INFO  (jetty-closer-213-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 135702 INFO  (jetty-closer-213-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@63f150d: rootName = solr_55232, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4efd1bc3
   [junit4]   2> 135704 INFO  (jetty-closer-213-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 135704 INFO  (jetty-closer-213-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7b95e4fe: rootName = solr_55235, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@21a482bd
   [junit4]   2> 135704 INFO  (jetty-closer-213-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 135705 INFO  (jetty-closer-213-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 135705 INFO  (jetty-closer-213-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1acc751f: rootName = solr_55232, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@21a482bd
   [junit4]   2> 135706 INFO  (jetty-closer-213-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 135709 INFO  (closeThreadPool-218-thread-2) [    ] o.a.s.c.Overseer Overseer (id=72107182094483462-127.0.0.1:55235_solr-n_0000000000) closing
   [junit4]   2> 135709 INFO  (OverseerStateUpdate-72107182094483462-127.0.0.1:55235_solr-n_0000000000) [n:127.0.0.1:55235_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:55235_solr
   [junit4]   2> 135713 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1002d199e7f0006, likely client has closed socket
   [junit4]   2> 135717 INFO  (OverseerAutoScalingTriggerThread-72107182094483462-127.0.0.1:55235_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 135718 INFO  (closeThreadPool-218-thread-1) [    ] o.a.s.c.Overseer Overseer (id=72107182094483462-127.0.0.1:55235_solr-n_0000000000) closing
   [junit4]   2> 135719 INFO  (zkCallback-179-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
   [junit4]   2> 135719 INFO  (jetty-closer-213-thread-2) [    ] o.a.s.c.Overseer Overseer (id=72107182094483462-127.0.0.1:55235_solr-n_0000000000) closing
   [junit4]   2> 135724 INFO  (jetty-closer-213-thread-2) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@5ecfdeda{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 135725 INFO  (jetty-closer-213-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@76d00d75{/solr,null,UNAVAILABLE}
   [junit4]   2> 135725 INFO  (jetty-closer-213-thread-2) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 135725 INFO  (jetty-closer-213-thread-1) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@4ad52d46{HTTP/1.1,[http/1.1]}{127.0.0.1:55232}
   [junit4]   2> 135726 INFO  (jetty-closer-213-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@641f1ada{/solr,null,UNAVAILABLE}
   [junit4]   2> 135727 INFO  (jetty-closer-213-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 135729 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 135730 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1002d199e7f0000, likely client has closed socket
   [junit4]   2> 135732 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1002d199e7f0001, likely client has closed socket
   [junit4]   2> 135733 WARN  (ZkTestServer Run Thread) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	11	/solr/collections/collection2/terms/shard1
   [junit4]   2> 	7	/solr/aliases.json
   [junit4]   2> 	7	/solr/clusterprops.json
   [junit4]   2> 	4	/solr/collections/collection2/state.json
   [junit4]   2> 	4	/solr/configs/config
   [junit4]   2> 	3	/solr/security.json
   [junit4]   2> 	3	/solr/collections/collection1/state.json
   [junit4]   2> 	3	/solr/collections/collection1/terms/shard1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	16	/solr/collections/collection2/state.json
   [junit4]   2> 	7	/solr/clusterstate.json
   [junit4]   2> 	2	/solr/overseer_elect/election/72107182094483462-127.0.0.1:55235_solr-n_0000000000
   [junit4]   2> 	2	/solr/collections/collection1/state.json
   [junit4]   2> 	2	/solr/collections/collection2/leader_elect/shard1/election/72107182094483462-core_node3-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	7	/solr/collections
   [junit4]   2> 	5	/solr/live_nodes
   [junit4]   2> 
   [junit4]   2> 135737 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:55221
   [junit4]   2> 135737 INFO  (SUITE-LeaderTragicEventTest-seed#[179C25DD2B62C249]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 55221
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderTragicEventTest_179C25DD2B62C249-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {_version_=PostingsFormat(name=Memory), id=PostingsFormat(name=Direct)}, docValues:{_version_=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=1868, maxMBSortInHeap=6.020197949561204, sim=RandomSimilarity(queryNorm=false): {}, locale=en-UG, timezone=Asia/Ust-Nera
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 9.0.4 (64-bit)/cpus=3,threads=1,free=53936104,total=175112192
   [junit4]   2> NOTE: All tests run in this JVM: [LeaderTragicEventTest, LeaderTragicEventTest, LeaderTragicEventTest]
   [junit4] Completed [5/5 (1!)] on J0 in 48.32s, 2 tests, 1 error <<< FAILURES!

[...truncated 10 lines...]
BUILD FAILED
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\common-build.xml:1572: The following error occurred while executing this line:
C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows\lucene\common-build.xml:1099: There were test failures: 5 suites, 10 tests, 1 error [seed: 179C25DD2B62C249]

Total time: 2 minutes 20 seconds

[repro] Setting last failure code to 1

[repro] Failures:
[repro]   1/5 failed: org.apache.solr.cloud.LeaderTragicEventTest
[repro] Exiting with code 1

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows>IF EXIST lucene\build move /y lucene\build lucene\build.repro 
        1 dir(s) moved.

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows>IF EXIST solr\build move /y solr\build solr\build.repro 
        1 dir(s) moved.

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows>IF EXIST lucene\build.orig move /y lucene\build.orig lucene\build 
        1 dir(s) moved.

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows>IF EXIST solr\build.orig move /y solr\build.orig solr\build 
        1 dir(s) moved.

C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows>exit 0 
Archiving artifacts
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
No credentials specified
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=9f9d65d6ec9b54bf903702620c41acc75b481809, workspace=C:\Users\jenkins\workspace\Lucene-Solr-7.x-Windows
[WARNINGS] Computing warning deltas based on reference build #1014
Recording test results
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2