You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2020/10/09 06:44:38 UTC

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-12.0.2) - Build # 6610 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6610/
Java: 64bit/jdk-12.0.2 -XX:-UseCompressedOops -XX:+UseParallelGC

3 tests failed.
FAILED:  org.apache.solr.handler.TestSolrConfigHandlerCloud.test

Error Message:
java.lang.AssertionError: {
  "responseHeader":{
    "status":500,
    "QTime":30131},
  "errorMessages":["4 out of 5 the property overlay to be of version 3 within 30 seconds! Failed cores: [http://127.0.0.1:63166/zahww/cq/collection1_shard2_replica_n2/, http://127.0.0.1:63194/zahww/cq/collection1_shard2_replica_n5/, http://127.0.0.1:63179/zahww/cq/collection1_shard1_replica_n1/, http://127.0.0.1:63215/zahww/cq/collection1_shard1_replica_n6/]\n"],
  "WARNING":"This response format is experimental.  It is likely to change in the future.",
  "error":{
    "metadata":[
      "error-class","org.apache.solr.common.SolrException",
      "root-error-class","org.apache.solr.common.SolrException"],
    "msg":"4 out of 5 the property overlay to be of version 3 within 30 seconds! Failed cores: [http://127.0.0.1:63166/zahww/cq/collection1_shard2_replica_n2/, http://127.0.0.1:63194/zahww/cq/collection1_shard2_replica_n5/, http://127.0.0.1:63179/zahww/cq/collection1_shard1_replica_n1/, http://127.0.0.1:63215/zahww/cq/collection1_shard1_replica_n6/]",
    "trace":"org.apache.solr.common.SolrException: 4 out of 5 the property overlay to be of version 3 within 30 seconds! Failed cores: [http://127.0.0.1:63166/zahww/cq/collection1_shard2_replica_n2/, http://127.0.0.1:63194/zahww/cq/collection1_shard2_replica_n5/, http://127.0.0.1:63179/zahww/cq/collection1_shard1_replica_n1/, http://127.0.0.1:63215/zahww/cq/collection1_shard1_replica_n6/]\n\tat org.apache.solr.handler.SolrConfigHandler.waitForAllReplicasState(SolrConfigHandler.java:827)\n\tat org.apache.solr.handler.SolrConfigHandler$Command.handleCommands(SolrConfigHandler.java:544)\n\tat org.apache.solr.handler.SolrConfigHandler$Command.handlePOST(SolrConfigHandler.java:376)\n\tat org.apache.solr.handler.SolrConfigHandler.handleRequestBody(SolrConfigHandler.java:138)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:2610)\n\tat org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:451)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:365)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)\n\tat org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)\n\tat org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:500)\n\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)\n\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)\n\tat org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)\n\tat java.base/java.lang.Thread.run(Thread.java:835)\n",
    "code":500}}
 expected null, but was:<[4 out of 5 the property overlay to be of version 3 within 30 seconds! Failed cores: [http://127.0.0.1:63166/zahww/cq/collection1_shard2_replica_n2/, http://127.0.0.1:63194/zahww/cq/collection1_shard2_replica_n5/, http://127.0.0.1:63179/zahww/cq/collection1_shard1_replica_n1/, http://127.0.0.1:63215/zahww/cq/collection1_shard1_replica_n6/]
]>

Stack Trace:
java.lang.AssertionError: {
  "responseHeader":{
    "status":500,
    "QTime":30131},
  "errorMessages":["4 out of 5 the property overlay to be of version 3 within 30 seconds! Failed cores: [http://127.0.0.1:63166/zahww/cq/collection1_shard2_replica_n2/, http://127.0.0.1:63194/zahww/cq/collection1_shard2_replica_n5/, http://127.0.0.1:63179/zahww/cq/collection1_shard1_replica_n1/, http://127.0.0.1:63215/zahww/cq/collection1_shard1_replica_n6/]\n"],
  "WARNING":"This response format is experimental.  It is likely to change in the future.",
  "error":{
    "metadata":[
      "error-class","org.apache.solr.common.SolrException",
      "root-error-class","org.apache.solr.common.SolrException"],
    "msg":"4 out of 5 the property overlay to be of version 3 within 30 seconds! Failed cores: [http://127.0.0.1:63166/zahww/cq/collection1_shard2_replica_n2/, http://127.0.0.1:63194/zahww/cq/collection1_shard2_replica_n5/, http://127.0.0.1:63179/zahww/cq/collection1_shard1_replica_n1/, http://127.0.0.1:63215/zahww/cq/collection1_shard1_replica_n6/]",
    "trace":"org.apache.solr.common.SolrException: 4 out of 5 the property overlay to be of version 3 within 30 seconds! Failed cores: [http://127.0.0.1:63166/zahww/cq/collection1_shard2_replica_n2/, http://127.0.0.1:63194/zahww/cq/collection1_shard2_replica_n5/, http://127.0.0.1:63179/zahww/cq/collection1_shard1_replica_n1/, http://127.0.0.1:63215/zahww/cq/collection1_shard1_replica_n6/]\n\tat org.apache.solr.handler.SolrConfigHandler.waitForAllReplicasState(SolrConfigHandler.java:827)\n\tat org.apache.solr.handler.SolrConfigHandler$Command.handleCommands(SolrConfigHandler.java:544)\n\tat org.apache.solr.handler.SolrConfigHandler$Command.handlePOST(SolrConfigHandler.java:376)\n\tat org.apache.solr.handler.SolrConfigHandler.handleRequestBody(SolrConfigHandler.java:138)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:2610)\n\tat org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:451)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:365)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)\n\tat org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)\n\tat org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:500)\n\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)\n\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)\n\tat org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)\n\tat java.base/java.lang.Thread.run(Thread.java:835)\n",
    "code":500}}
 expected null, but was:<[4 out of 5 the property overlay to be of version 3 within 30 seconds! Failed cores: [http://127.0.0.1:63166/zahww/cq/collection1_shard2_replica_n2/, http://127.0.0.1:63194/zahww/cq/collection1_shard2_replica_n5/, http://127.0.0.1:63179/zahww/cq/collection1_shard1_replica_n1/, http://127.0.0.1:63215/zahww/cq/collection1_shard1_replica_n6/]
]>
	at __randomizedtesting.SeedInfo.seed([C121695712E93AA0:4975568DBC155758]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotNull(Assert.java:755)
	at org.junit.Assert.assertNull(Assert.java:737)
	at org.apache.solr.core.TestSolrConfigHandler.runConfigCommand(TestSolrConfigHandler.java:223)
	at org.apache.solr.core.TestSolrConfigHandler.reqhandlertests(TestSolrConfigHandler.java:308)
	at org.apache.solr.handler.TestSolrConfigHandlerCloud.testReqHandlerAPIs(TestSolrConfigHandlerCloud.java:80)
	at org.apache.solr.handler.TestSolrConfigHandlerCloud.test(TestSolrConfigHandlerCloud.java:50)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1090)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1061)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:835)


FAILED:  org.apache.solr.handler.TestSystemCollAutoCreate.testAutoCreate

Error Message:
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:62177: Underlying core creation failed while creating collection: control_collection

Stack Trace:
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:62177: Underlying core creation failed while creating collection: control_collection
	at __randomizedtesting.SeedInfo.seed([C121695712E93AA0:FE7ADD86A497F3CD]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:698)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:269)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
	at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:390)
	at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:360)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1170)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:931)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:865)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:229)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:246)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:338)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1088)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1061)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:835)


FAILED:  org.apache.solr.handler.component.CustomHighlightComponentTest.test

Error Message:
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException: Error from server at https://127.0.0.1:63092/solr/collection28: error processing commands

Stack Trace:
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException: Error from server at https://127.0.0.1:63092/solr/collection28: error processing commands
	at __randomizedtesting.SeedInfo.seed([C121695712E93AA0:4975568DBC155758]:0)
	at org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException.create(BaseHttpSolrClient.java:67)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:664)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:269)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
	at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:390)
	at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:360)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1170)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:931)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:865)
	at org.apache.solr.handler.component.CustomHighlightComponentTest.test(CustomHighlightComponentTest.java:163)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:835)




Build Log:
[...truncated 7785 lines...]
  2> 376228 ERROR (qtp1319016379-1205) [n:127.0.0.1:63092_solr c:collection28 s:shard3 r:core_node11 x:collection28_shard3_replica_n8 ] o.a.s.h.SolrConfigHandler ERROR:[{
  2>     "add-requesthandler":{
  2>       "name":"/custom_select288363682",
  2>       "class":"org.apache.solr.handler.component.SearchHandler",
  2>       "components":[
  2>         "query",
  2>         "highlight"]},
  2>     "errorMessages":[" '/custom_select288363682' already exists . Do an 'update-requesthandler' , if you want to change it "]}]
  2> 376228 INFO  (qtp1319016379-1205) [n:127.0.0.1:63092_solr c:collection28 s:shard3 r:core_node11 x:collection28_shard3_replica_n8 ] o.a.s.c.S.Request [collection28_shard3_replica_n8]  webapp=/solr path=/config params={wt=javabin&version=2} status=400 QTime=38
  2> 376231 INFO  (TEST-CustomHighlightComponentTest.test-seed#[C121695712E93AA0]) [     ] o.a.s.c.s.i.BaseCloudSolrClient request was not communication error it seems
  2> 376231 INFO  (TEST-CustomHighlightComponentTest.test-seed#[C121695712E93AA0]) [     ] o.a.s.c.s.i.BaseCloudSolrClient Request to collection [collection28] failed due to (400) org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException: Error from server at https://127.0.0.1:63092/solr/collection28: error processing commands, retry=0 maxRetries=5 commError=false errorCode=400 
  2> 376232 INFO  (TEST-CustomHighlightComponentTest.test-seed#[C121695712E93AA0]) [     ] o.a.s.SolrTestCaseJ4 ###Ending test
   >     org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException: Error from server at https://127.0.0.1:63092/solr/collection28: error processing commands
   >         at __randomizedtesting.SeedInfo.seed([C121695712E93AA0:4975568DBC155758]:0)
   >         at org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException.create(BaseHttpSolrClient.java:67)
   >         at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:664)
   >         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:269)
   >         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
   >         at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:390)
   >         at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:360)
   >         at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1170)
   >         at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:931)
   >         at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:865)
   >         at org.apache.solr.handler.component.CustomHighlightComponentTest.test(CustomHighlightComponentTest.java:163)
   >         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   >         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   >         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   >         at java.base/java.lang.reflect.Method.invoke(Method.java:567)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
   >         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   >         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   >         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   >         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   >         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   >         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   >         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   >         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   >         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   >         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   >         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   >         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
   >         at java.base/java.lang.Thread.run(Thread.java:835)
  2> NOTE: reproduce with: gradlew test --tests CustomHighlightComponentTest.test -Dtests.seed=C121695712E93AA0 -Dtests.slow=true -Dtests.locale=en-IN -Dtests.timezone=PST8PDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
  2> 376377 INFO  (searcherExecutor-778-thread-1-processing-n:127.0.0.1:63085_solr) [n:127.0.0.1:63085_solr     ] o.a.s.c.SolrCore [collection28_shard2_replica_n6]  Registered new searcher autowarm time: 0 ms
  2> 376381 INFO  (Thread-403) [n:127.0.0.1:63085_solr     ] o.a.s.c.SolrCore [collection28_shard2_replica_n6]  CLOSING SolrCore org.apache.solr.core.SolrCore@2038f22c
  2> 376381 INFO  (Thread-403) [n:127.0.0.1:63085_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection28.shard2.replica_n6 tag=SolrCore@2038f22c
  2> 376381 INFO  (Thread-403) [n:127.0.0.1:63085_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4068eebf: rootName = solr_63085, domain = solr.core.collection28.shard2.replica_n6, service url = null, agent id = null] for registry solr.core.collection28.shard2.replica_n6/com.codahale.metrics.MetricRegistry@1e9fa708
  2> 376426 INFO  (jetty-closer-1846-thread-2) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1903314491
  2> 376426 INFO  (jetty-closer-1846-thread-2) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:63084_solr
  2> 376426 INFO  (jetty-closer-1846-thread-3) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1889042401
  2> 376426 INFO  (jetty-closer-1846-thread-3) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:63086_solr
  2> 376426 INFO  (jetty-closer-1846-thread-4) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=457617831
  2> 376426 INFO  (jetty-closer-1846-thread-4) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:63085_solr
  2> 376429 INFO  (jetty-closer-1846-thread-5) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2119001738
  2> 376429 INFO  (jetty-closer-1846-thread-5) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:63087_solr
  2> 376430 INFO  (jetty-closer-1846-thread-2) [     ] o.a.s.c.ZkController Publish this node as DOWN...
  2> 376430 INFO  (jetty-closer-1846-thread-2) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:63084_solr as DOWN
  2> 376432 INFO  (jetty-closer-1846-thread-4) [     ] o.a.s.c.ZkController Publish this node as DOWN...
  2> 376432 INFO  (jetty-closer-1846-thread-4) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:63085_solr as DOWN
  2> 376436 INFO  (jetty-closer-1846-thread-6) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=385316991
  2> 376436 INFO  (jetty-closer-1846-thread-6) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:63092_solr
  2> 376497 INFO  (zkCallback-627-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (3)
  2> 376497 INFO  (zkCallback-679-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (3)
  2> 376497 INFO  (zkCallback-683-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (1)
  2> 376497 INFO  (jetty-closer-1846-thread-3) [     ] o.a.s.c.ZkController Publish this node as DOWN...
  2> 376497 INFO  (jetty-closer-1846-thread-3) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:63086_solr as DOWN
  2> 376498 INFO  (zkCallback-647-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (3)
  2> 376499 INFO  (zkCallback-663-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (1)
  2> 376512 INFO  (jetty-closer-1846-thread-6) [     ] o.a.s.c.ZkController Publish this node as DOWN...
  2> 376512 INFO  (jetty-closer-1846-thread-6) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:63092_solr as DOWN
  2> 376513 INFO  (zkCallback-644-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (2)
  2> 376516 INFO  (zkCallback-627-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1)
  2> 376561 INFO  (coreCloseExecutor-1859-thread-1) [n:127.0.0.1:63085_solr     ] o.a.s.c.SolrCore [collection28_shard2_replica_n6]  CLOSING SolrCore org.apache.solr.core.SolrCore@6c19f4be
  2> 376564 INFO  (jetty-closer-1846-thread-5) [     ] o.a.s.c.ZkController Publish this node as DOWN...
  2> 376564 INFO  (jetty-closer-1846-thread-5) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:63087_solr as DOWN
  2> 376567 INFO  (zkCallback-679-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1)
  2> 376569 INFO  (zkCallback-647-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1)
  2> 376587 INFO  (zkCallback-644-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
  2> 376593 INFO  (coreCloseExecutor-1861-thread-1) [n:127.0.0.1:63084_solr     ] o.a.s.c.SolrCore [collection28_shard3_replica_n10]  CLOSING SolrCore org.apache.solr.core.SolrCore@50321c49
  2> 376593 INFO  (coreCloseExecutor-1861-thread-1) [n:127.0.0.1:63084_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection28.shard3.replica_n10 tag=SolrCore@50321c49
  2> 376593 INFO  (coreCloseExecutor-1861-thread-1) [n:127.0.0.1:63084_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5216f1ff: rootName = solr_63084, domain = solr.core.collection28.shard3.replica_n10, service url = null, agent id = null] for registry solr.core.collection28.shard3.replica_n10/com.codahale.metrics.MetricRegistry@5694e24
  2> 376617 INFO  (coreCloseExecutor-1863-thread-1) [n:127.0.0.1:63086_solr     ] o.a.s.c.SolrCore [collection28_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@6ba195ed
  2> 376617 INFO  (coreCloseExecutor-1863-thread-1) [n:127.0.0.1:63086_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection28.shard1.replica_n1 tag=SolrCore@6ba195ed
  2> 376617 INFO  (coreCloseExecutor-1863-thread-1) [n:127.0.0.1:63086_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1ca194f2: rootName = solr_63086, domain = solr.core.collection28.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection28.shard1.replica_n1/com.codahale.metrics.MetricRegistry@249c64eb
  2> 376635 INFO  (coreCloseExecutor-1865-thread-1) [n:127.0.0.1:63087_solr     ] o.a.s.c.SolrCore [collection28_shard2_replica_n4]  CLOSING SolrCore org.apache.solr.core.SolrCore@4d1ca375
  2> 376635 INFO  (coreCloseExecutor-1865-thread-1) [n:127.0.0.1:63087_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection28.shard2.replica_n4 tag=SolrCore@4d1ca375
  2> 376635 INFO  (coreCloseExecutor-1865-thread-1) [n:127.0.0.1:63087_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4585479c: rootName = solr_63087, domain = solr.core.collection28.shard2.replica_n4, service url = null, agent id = null] for registry solr.core.collection28.shard2.replica_n4/com.codahale.metrics.MetricRegistry@386729c0
  2> 376656 INFO  (jetty-closer-1846-thread-1) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=116101583
  2> 376656 INFO  (jetty-closer-1846-thread-1) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:63088_solr
  2> 376656 INFO  (coreCloseExecutor-1867-thread-1) [n:127.0.0.1:63092_solr     ] o.a.s.c.SolrCore [collection28_shard3_replica_n8]  CLOSING SolrCore org.apache.solr.core.SolrCore@16ae643d
  2> 376657 INFO  (coreCloseExecutor-1867-thread-1) [n:127.0.0.1:63092_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection28.shard3.replica_n8 tag=SolrCore@16ae643d
  2> 376657 INFO  (coreCloseExecutor-1867-thread-1) [n:127.0.0.1:63092_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@286e3aa4: rootName = solr_63092, domain = solr.core.collection28.shard3.replica_n8, service url = null, agent id = null] for registry solr.core.collection28.shard3.replica_n8/com.codahale.metrics.MetricRegistry@3a988fa
  2> 376658 INFO  (zkCallback-644-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [1])
  2> 376660 INFO  (zkCallback-683-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [1])
  2> 376660 INFO  (zkCallback-683-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [1])
  2> 376661 INFO  (zkCallback-663-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [1])
  2> 376661 INFO  (jetty-closer-1846-thread-1) [     ] o.a.s.c.ZkController Publish this node as DOWN...
  2> 376661 INFO  (zkCallback-663-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [1])
  2> 376661 INFO  (jetty-closer-1846-thread-1) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:63088_solr as DOWN
  2> 376662 INFO  (zkCallback-627-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [1])
  2> 376662 INFO  (zkCallback-647-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [1])
  2> 376662 INFO  (zkCallback-647-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [1])
  2> 376662 INFO  (zkCallback-644-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [1])
  2> 376663 INFO  (zkCallback-679-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [1])
  2> 376663 INFO  (zkCallback-679-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [1])
  2> 376669 INFO  (zkCallback-663-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
  2> 376670 INFO  (zkCallback-683-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
  2> 376673 INFO  (zkCallback-627-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
  2> 376676 INFO  (zkCallback-644-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
  2> 376679 INFO  (zkCallback-647-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
  2> 376687 INFO  (zkCallback-679-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
  2> 376705 INFO  (coreCloseExecutor-1870-thread-1) [n:127.0.0.1:63088_solr     ] o.a.s.c.SolrCore [collection28_shard1_replica_n3]  CLOSING SolrCore org.apache.solr.core.SolrCore@60f816f9
  2> 376706 INFO  (coreCloseExecutor-1870-thread-1) [n:127.0.0.1:63088_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection28.shard1.replica_n3 tag=SolrCore@60f816f9
  2> 376706 INFO  (coreCloseExecutor-1870-thread-1) [n:127.0.0.1:63088_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5b13934a: rootName = solr_63088, domain = solr.core.collection28.shard1.replica_n3, service url = null, agent id = null] for registry solr.core.collection28.shard1.replica_n3/com.codahale.metrics.MetricRegistry@61f0dde2
  2> 376857 INFO  (zkCallback-644-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [0])
  2> 376857 INFO  (zkCallback-627-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [0])
  2> 376857 INFO  (zkCallback-644-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [0])
  2> 376857 INFO  (zkCallback-647-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [0])
  2> 376857 INFO  (zkCallback-647-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [0])
  2> 376858 INFO  (zkCallback-679-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [0])
  2> 376859 INFO  (zkCallback-683-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [0])
  2> 376859 INFO  (zkCallback-683-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [0])
  2> 376860 INFO  (zkCallback-679-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [0])
  2> 376865 INFO  (zkCallback-663-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [0])
  2> 376865 INFO  (zkCallback-663-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection28/state.json] for collection [collection28] has occurred - updating... (live nodes size: [0])
  2> 377207 INFO  (searcherExecutor-778-thread-1-processing-n:127.0.0.1:63085_solr) [n:127.0.0.1:63085_solr     ] o.a.s.c.SolrCore [collection28_shard2_replica_n6]  Registered new searcher autowarm time: 0 ms
  2> 378074 INFO  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
  2> 378074 INFO  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
  2> 378075 INFO  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
  2> 378075 INFO  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
  2> 378079 INFO  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
  2> 378079 INFO  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
  2> 378085 INFO  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
  2> 378085 INFO  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
  2> 378088 INFO  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
  2> 378088 INFO  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
  2> 378232 INFO  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
  2> 378236 INFO  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
  2> 378236 INFO  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
  2> 378236 INFO  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1680052110413004800
  2> 378240 INFO  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
  2> 378254 INFO  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
  2> 378255 INFO  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
  2> 378256 INFO  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
  2> 378256 INFO  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1680052110433976320
  2> 378270 INFO  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
  2> 378282 INFO  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
  2> 378284 INFO  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
  2> 378284 INFO  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
  2> 378284 INFO  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1680052110463336448
  2> 378291 INFO  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
  2> 378342 INFO  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
  2> 378343 INFO  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
  2> 378346 INFO  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
  2> 378346 INFO  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1680052110528348160
  2> 378347 INFO  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
  2> 378355 INFO  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
  2> 378355 INFO  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
  2> 378355 INFO  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1680052110537785344
  2> 378362 INFO  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
  2> 378363 INFO  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
  2> 378424 INFO  (searcherExecutor-780-thread-1-processing-n:127.0.0.1:63092_solr) [n:127.0.0.1:63092_solr     ] o.a.s.c.SolrCore [collection28_shard3_replica_n8]  Registered new searcher autowarm time: 0 ms
  2> 378425 INFO  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.c.SolrCore [collection28_shard3_replica_n8]  CLOSING SolrCore org.apache.solr.core.SolrCore@3af8859d
  2> 378428 INFO  (searcherExecutor-783-thread-1-processing-n:127.0.0.1:63088_solr) [n:127.0.0.1:63088_solr     ] o.a.s.c.SolrCore [collection28_shard1_replica_n3]  Registered new searcher autowarm time: 0 ms
  2> 378428 INFO  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.c.SolrCore [collection28_shard1_replica_n3]  CLOSING SolrCore org.apache.solr.core.SolrCore@4f745859
  2> 378442 INFO  (searcherExecutor-790-thread-1-processing-n:127.0.0.1:63084_solr) [n:127.0.0.1:63084_solr     ] o.a.s.c.SolrCore [collection28_shard3_replica_n10]  Registered new searcher autowarm time: 0 ms
  2> 378444 INFO  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.c.SolrCore [collection28_shard3_replica_n10]  CLOSING SolrCore org.apache.solr.core.SolrCore@229c4fdf
  2> 378453 INFO  (searcherExecutor-793-thread-1-processing-n:127.0.0.1:63087_solr) [n:127.0.0.1:63087_solr     ] o.a.s.c.SolrCore [collection28_shard2_replica_n4]  Registered new searcher autowarm time: 0 ms
  2> 378454 INFO  (searcherExecutor-795-thread-1-processing-n:127.0.0.1:63086_solr) [n:127.0.0.1:63086_solr     ] o.a.s.c.SolrCore [collection28_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
  2> 378456 INFO  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.c.SolrCore [collection28_shard2_replica_n4]  CLOSING SolrCore org.apache.solr.core.SolrCore@1ae9e6fa
  2> 378457 INFO  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.c.SolrCore [collection28_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@55e49064
  2> 378625 INFO  (searcherExecutor-780-thread-1-processing-n:127.0.0.1:63092_solr) [n:127.0.0.1:63092_solr     ] o.a.s.c.SolrCore [collection28_shard3_replica_n8]  Registered new searcher autowarm time: 0 ms
  2> 378634 INFO  (searcherExecutor-783-thread-1-processing-n:127.0.0.1:63088_solr) [n:127.0.0.1:63088_solr     ] o.a.s.c.SolrCore [collection28_shard1_replica_n3]  Registered new searcher autowarm time: 0 ms
  2> 378655 INFO  (searcherExecutor-790-thread-1-processing-n:127.0.0.1:63084_solr) [n:127.0.0.1:63084_solr     ] o.a.s.c.SolrCore [collection28_shard3_replica_n10]  Registered new searcher autowarm time: 0 ms
  2> 378705 INFO  (searcherExecutor-795-thread-1-processing-n:127.0.0.1:63086_solr) [n:127.0.0.1:63086_solr     ] o.a.s.c.SolrCore [collection28_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
  2> 378705 INFO  (searcherExecutor-793-thread-1-processing-n:127.0.0.1:63087_solr) [n:127.0.0.1:63087_solr     ] o.a.s.c.SolrCore [collection28_shard2_replica_n4]  Registered new searcher autowarm time: 0 ms
  2> 379475 INFO  (coreCloseExecutor-1861-thread-1) [n:127.0.0.1:63084_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection28.shard3.leader tag=SolrCore@50321c49
  2> 379475 INFO  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection28.shard3.replica_n10 tag=SolrCore@229c4fdf
  2> 379475 INFO  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@12aab730: rootName = solr_63084, domain = solr.core.collection28.shard3.replica_n10, service url = null, agent id = null] for registry solr.core.collection28.shard3.replica_n10/com.codahale.metrics.MetricRegistry@5694e24
  2> 379504 INFO  (Thread-403) [n:127.0.0.1:63085_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection28.shard2.leader tag=SolrCore@2038f22c
  2> 379504 INFO  (coreCloseExecutor-1859-thread-1) [n:127.0.0.1:63085_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection28.shard2.replica_n6 tag=SolrCore@6c19f4be
  2> 379504 INFO  (coreCloseExecutor-1865-thread-1) [n:127.0.0.1:63087_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection28.shard2.leader tag=SolrCore@4d1ca375
  2> 379504 INFO  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection28.shard2.replica_n4 tag=SolrCore@1ae9e6fa
  2> 379504 INFO  (coreCloseExecutor-1859-thread-1) [n:127.0.0.1:63085_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3f5c2f0a: rootName = solr_63085, domain = solr.core.collection28.shard2.replica_n6, service url = null, agent id = null] for registry solr.core.collection28.shard2.replica_n6/com.codahale.metrics.MetricRegistry@1e9fa708
  2> 379504 INFO  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3f4e6bd6: rootName = solr_63087, domain = solr.core.collection28.shard2.replica_n4, service url = null, agent id = null] for registry solr.core.collection28.shard2.replica_n4/com.codahale.metrics.MetricRegistry@386729c0
  2> 379506 INFO  (coreCloseExecutor-1867-thread-1) [n:127.0.0.1:63092_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection28.shard3.leader tag=SolrCore@16ae643d
  2> 379506 INFO  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection28.shard3.replica_n8 tag=SolrCore@3af8859d
  2> 379506 INFO  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@34de6999: rootName = solr_63092, domain = solr.core.collection28.shard3.replica_n8, service url = null, agent id = null] for registry solr.core.collection28.shard3.replica_n8/com.codahale.metrics.MetricRegistry@3a988fa
  2> 379508 INFO  (coreCloseExecutor-1863-thread-1) [n:127.0.0.1:63086_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection28.shard1.leader tag=SolrCore@6ba195ed
  2> 379508 INFO  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection28.shard1.replica_n1 tag=SolrCore@55e49064
  2> 379508 INFO  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@445da074: rootName = solr_63086, domain = solr.core.collection28.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection28.shard1.replica_n1/com.codahale.metrics.MetricRegistry@249c64eb
  2> 379528 INFO  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection28.shard1.replica_n3 tag=SolrCore@4f745859
  2> 379528 INFO  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3bb6446a: rootName = solr_63088, domain = solr.core.collection28.shard1.replica_n3, service url = null, agent id = null] for registry solr.core.collection28.shard1.replica_n3/com.codahale.metrics.MetricRegistry@61f0dde2
  2> 381213 INFO  (coreCloseExecutor-1859-thread-1) [n:127.0.0.1:63085_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection28.shard2.leader tag=SolrCore@6c19f4be
  2> 381213 INFO  (coreCloseExecutor-1859-thread-1) [n:127.0.0.1:63085_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
  2> 381225 INFO  (jetty-closer-1846-thread-4) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
  2> 381225 INFO  (jetty-closer-1846-thread-4) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@10592242: rootName = solr_63085, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@448ce090
  2> 384308 INFO  (jetty-closer-1846-thread-4) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
  2> 384308 INFO  (jetty-closer-1846-thread-4) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@38d1940d: rootName = solr_63085, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@58f758ac
  2> 387166 INFO  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection28.shard3.leader tag=SolrCore@229c4fdf
  2> 387167 INFO  (jetty-closer-1846-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
  2> 387167 INFO  (jetty-closer-1846-thread-2) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6d044e6f: rootName = solr_63084, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@ecb2002
  2> 387174 INFO  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
  2> 387398 WARN  (Thread-405) [n:127.0.0.1:63084_solr     ] o.a.s.c.ZkController listener throws error
  2>           => org.apache.solr.common.SolrException: Unable to reload core [collection28_shard3_replica_n10]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1671)
  2> org.apache.solr.common.SolrException: Unable to reload core [collection28_shard3_replica_n10]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1671) ~[main/:?]
  2> 	at org.apache.solr.core.SolrCore.lambda$getConfListener$20(SolrCore.java:3110) ~[main/:?]
  2> 	at org.apache.solr.cloud.ZkController.lambda$fireEventListeners$14(ZkController.java:2500) [main/:9.0.0-SNAPSHOT 08e38d3452d548189edba691c2853b63b1fa55ae [snapshot build, details omitted]]
  2> 	at java.lang.Thread.run(Thread.java:835) [?:?]
  2> Caused by: java.lang.IllegalStateException: This CoreContainer has been closed
  2> 	at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:1175) ~[main/:?]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1632) ~[main/:?]
  2> 	... 3 more
  2> 388267 INFO  (jetty-closer-1846-thread-4) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
  2> 388267 INFO  (jetty-closer-1846-thread-4) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7746841c: rootName = solr_63085, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@58c4eae6
  2> 388267 INFO  (jetty-closer-1846-thread-4) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
  2> 388840 INFO  (jetty-closer-1846-thread-4) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@5e6b41e4{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:0}
  2> 388846 INFO  (jetty-closer-1846-thread-4) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@19ce1393{/solr,null,UNAVAILABLE}
  2> 388848 INFO  (jetty-closer-1846-thread-4) [     ] o.e.j.s.session node0 Stopped scavenging
  2> 389514 WARN  (jetty-closer-1846-thread-5) [     ] o.a.s.m.SolrMetricManager Could not obtain lock to modify reporters registry: solr.node
  2> 389517 WARN  (jetty-closer-1846-thread-3) [     ] o.a.s.m.SolrMetricManager Could not obtain lock to modify reporters registry: solr.node
  2> 389546 WARN  (jetty-closer-1846-thread-6) [     ] o.a.s.m.SolrMetricManager Could not obtain lock to modify reporters registry: solr.node
  2> 389568 WARN  (coreCloseExecutor-1870-thread-1) [n:127.0.0.1:63088_solr     ] o.a.s.m.SolrMetricManager Could not obtain lock to modify reporters registry: solr.collection.collection28.shard1.leader
  2> 392053 INFO  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection28.shard2.leader tag=SolrCore@1ae9e6fa
  2> 392053 INFO  (jetty-closer-1846-thread-5) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
  2> 392053 INFO  (jetty-closer-1846-thread-5) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@67fedfbe: rootName = solr_63087, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@58f758ac
  2> 392054 INFO  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
  2> 392059 WARN  (Thread-402) [n:127.0.0.1:63087_solr     ] o.a.s.c.ZkController listener throws error
  2>           => org.apache.solr.common.SolrException: Unable to reload core [collection28_shard2_replica_n4]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1671)
  2> org.apache.solr.common.SolrException: Unable to reload core [collection28_shard2_replica_n4]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1671) ~[main/:?]
  2> 	at org.apache.solr.core.SolrCore.lambda$getConfListener$20(SolrCore.java:3110) ~[main/:?]
  2> 	at org.apache.solr.cloud.ZkController.lambda$fireEventListeners$14(ZkController.java:2500) [main/:9.0.0-SNAPSHOT 08e38d3452d548189edba691c2853b63b1fa55ae [snapshot build, details omitted]]
  2> 	at java.lang.Thread.run(Thread.java:835) [?:?]
  2> Caused by: java.lang.IllegalStateException: This CoreContainer has been closed
  2> 	at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:1175) ~[main/:?]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1632) ~[main/:?]
  2> 	... 3 more
  2> 392096 INFO  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection28.shard3.leader tag=SolrCore@3af8859d
  2> 392096 INFO  (jetty-closer-1846-thread-6) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
  2> 392096 INFO  (jetty-closer-1846-thread-6) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@60cf527b: rootName = solr_63092, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@58f758ac
  2> 392097 INFO  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
  2> 392126 WARN  (Thread-406) [n:127.0.0.1:63092_solr     ] o.a.s.c.ZkController listener throws error
  2>           => org.apache.solr.common.SolrException: Unable to reload core [collection28_shard3_replica_n8]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1671)
  2> org.apache.solr.common.SolrException: Unable to reload core [collection28_shard3_replica_n8]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1671) ~[main/:?]
  2> 	at org.apache.solr.core.SolrCore.lambda$getConfListener$20(SolrCore.java:3110) ~[main/:?]
  2> 	at org.apache.solr.cloud.ZkController.lambda$fireEventListeners$14(ZkController.java:2500) [main/:9.0.0-SNAPSHOT 08e38d3452d548189edba691c2853b63b1fa55ae [snapshot build, details omitted]]
  2> 	at java.lang.Thread.run(Thread.java:835) [?:?]
  2> Caused by: java.lang.IllegalStateException: This CoreContainer has been closed
  2> 	at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:1175) ~[main/:?]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1632) ~[main/:?]
  2> 	... 3 more
  2> 392161 INFO  (jetty-closer-1846-thread-6) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
  2> 392161 INFO  (jetty-closer-1846-thread-6) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1b831f5c: rootName = solr_63092, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@58c4eae6
  2> 392318 INFO  (jetty-closer-1846-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
  2> 392319 INFO  (jetty-closer-1846-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@30033782: rootName = solr_63088, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@1433dda4
  2> 392589 INFO  (jetty-closer-1846-thread-6) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
  2> 392615 INFO  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection28.shard1.leader tag=SolrCore@55e49064
  2> 392616 INFO  (jetty-closer-1846-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
  2> 392616 INFO  (jetty-closer-1846-thread-3) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@44557589: rootName = solr_63086, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@58f758ac
  2> 392618 INFO  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
  2> 392646 WARN  (Thread-404) [n:127.0.0.1:63086_solr     ] o.a.s.c.ZkController listener throws error
  2>           => org.apache.solr.common.SolrException: Unable to reload core [collection28_shard1_replica_n1]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1671)
  2> org.apache.solr.common.SolrException: Unable to reload core [collection28_shard1_replica_n1]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1671) ~[main/:?]
  2> 	at org.apache.solr.core.SolrCore.lambda$getConfListener$20(SolrCore.java:3110) ~[main/:?]
  2> 	at org.apache.solr.cloud.ZkController.lambda$fireEventListeners$14(ZkController.java:2500) [main/:9.0.0-SNAPSHOT 08e38d3452d548189edba691c2853b63b1fa55ae [snapshot build, details omitted]]
  2> 	at java.lang.Thread.run(Thread.java:835) [?:?]
  2> Caused by: java.lang.IllegalStateException: This CoreContainer has been closed
  2> 	at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:1175) ~[main/:?]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1632) ~[main/:?]
  2> 	... 3 more
  2> 392750 INFO  (jetty-closer-1846-thread-6) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@3ecdb95a{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:0}
  2> 392751 INFO  (jetty-closer-1846-thread-6) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@48b4cb2d{/solr,null,UNAVAILABLE}
  2> 392755 INFO  (jetty-closer-1846-thread-6) [     ] o.e.j.s.session node0 Stopped scavenging
  2> 392890 INFO  (jetty-closer-1846-thread-5) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
  2> 392890 INFO  (jetty-closer-1846-thread-5) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3c932444: rootName = solr_63087, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@58c4eae6
  2> 393372 INFO  (jetty-closer-1846-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
  2> 393373 INFO  (jetty-closer-1846-thread-2) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4c6a56: rootName = solr_63084, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@58f758ac
  2> 393394 INFO  (jetty-closer-1846-thread-5) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
  2> 393419 INFO  (jetty-closer-1846-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
  2> 393419 INFO  (jetty-closer-1846-thread-3) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@cf6ad82: rootName = solr_63086, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@58c4eae6
  2> 393420 WARN  (jetty-launcher-606-thread-5-SendThread(127.0.0.1:63073)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x1005bfa047d000a.
  2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x1005bfa047d000a, likely server has closed socket
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
  2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x1005bfa047d000a, likely server has closed socket
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
  2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
  2> 393535 INFO  (jetty-closer-1846-thread-5) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@2060d340{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:0}
  2> 393538 INFO  (jetty-closer-1846-thread-5) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@226b9cf2{/solr,null,UNAVAILABLE}
  2> 393539 INFO  (jetty-closer-1846-thread-5) [     ] o.e.j.s.session node0 Stopped scavenging
  2> 393823 INFO  (jetty-closer-1846-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
  2> 393831 INFO  (closeThreadPool-1890-thread-1) [     ] o.a.s.c.Overseer Overseer (id=72158723413180428-127.0.0.1:63086_solr-n_0000000000) closing
  2> 393831 INFO  (OverseerStateUpdate-72158723413180428-127.0.0.1:63086_solr-n_0000000000) [n:127.0.0.1:63086_solr     ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:63086_solr
  2> 393836 INFO  (closeThreadPool-1890-thread-2) [     ] o.a.s.c.Overseer Overseer (id=72158723413180428-127.0.0.1:63086_solr-n_0000000000) closing
  2> 393878 INFO  (jetty-closer-1846-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
  2> 393878 INFO  (jetty-closer-1846-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4a15838c: rootName = solr_63088, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@58f758ac
  2> 393953 INFO  (jetty-closer-1846-thread-3) [     ] o.a.s.c.Overseer Overseer (id=72158723413180428-127.0.0.1:63086_solr-n_0000000000) closing
  2> 393955 INFO  (jetty-closer-1846-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
  2> 393955 INFO  (jetty-closer-1846-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
  2> 393955 INFO  (jetty-closer-1846-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1c4f28e3: rootName = solr_63088, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@58c4eae6
  2> 393955 INFO  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection28.shard1.leader tag=SolrCore@4f745859
  2> 393956 INFO  (jetty-closer-1846-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
  2> 393956 INFO  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
  2> 393956 INFO  (jetty-closer-1846-thread-2) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@56fefb2a: rootName = solr_63084, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@58c4eae6
  2> 393956 INFO  (jetty-closer-1846-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
  2> 393977 INFO  (jetty-closer-1846-thread-3) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@494ba57c{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:0}
  2> 393980 WARN  (Thread-401) [n:127.0.0.1:63088_solr     ] o.a.s.c.ZkController listener throws error
  2>           => org.apache.solr.common.SolrException: Unable to reload core [collection28_shard1_replica_n3]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1671)
  2> org.apache.solr.common.SolrException: Unable to reload core [collection28_shard1_replica_n3]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1671) ~[main/:?]
  2> 	at org.apache.solr.core.SolrCore.lambda$getConfListener$20(SolrCore.java:3110) ~[main/:?]
  2> 	at org.apache.solr.cloud.ZkController.lambda$fireEventListeners$14(ZkController.java:2500) [main/:9.0.0-SNAPSHOT 08e38d3452d548189edba691c2853b63b1fa55ae [snapshot build, details omitted]]
  2> 	at java.lang.Thread.run(Thread.java:835) [?:?]
  2> Caused by: java.lang.IllegalStateException: This CoreContainer has been closed
  2> 	at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:1175) ~[main/:?]
  2> 	at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:1632) ~[main/:?]
  2> 	... 3 more
  2> 393987 INFO  (jetty-closer-1846-thread-3) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@511df3cb{/solr,null,UNAVAILABLE}
  2> 393988 INFO  (jetty-closer-1846-thread-3) [     ] o.e.j.s.session node0 Stopped scavenging
  2> 394007 WARN  (jetty-launcher-606-thread-2-SendThread(127.0.0.1:63073)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x1005bfa047d000e.
  2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x1005bfa047d000e, likely server has closed socket
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
  2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x1005bfa047d000e, likely server has closed socket
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
  2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
  2> 394115 WARN  (jetty-launcher-606-thread-6-SendThread(127.0.0.1:63073)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x1005bfa047d000d.
  2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x1005bfa047d000d, likely server has closed socket
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
  2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x1005bfa047d000d, likely server has closed socket
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
  2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
  2> 394143 INFO  (jetty-closer-1846-thread-1) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@3bcd3b3e{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:0}
  2> 394147 INFO  (jetty-closer-1846-thread-1) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@62861517{/solr,null,UNAVAILABLE}
  2> 394149 INFO  (jetty-closer-1846-thread-1) [     ] o.e.j.s.session node0 Stopped scavenging
  2> 394249 INFO  (jetty-closer-1846-thread-2) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@23a3d103{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:0}
  2> 394257 INFO  (jetty-closer-1846-thread-2) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@58ba208{/solr,null,UNAVAILABLE}
  2> 394262 INFO  (jetty-closer-1846-thread-2) [     ] o.e.j.s.session node0 Stopped scavenging
  2> 394266 INFO  (SUITE-CustomHighlightComponentTest-seed#[C121695712E93AA0]-worker) [     ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
  2> 394295 WARN  (SUITE-CustomHighlightComponentTest-seed#[C121695712E93AA0]-worker-SendThread(127.0.0.1:63073)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x1005bfa047d0000.
  2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x1005bfa047d0000, likely server has closed socket
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
  2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x1005bfa047d0000, likely server has closed socket
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
  2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
  2> 394408 WARN  (SUITE-CustomHighlightComponentTest-seed#[C121695712E93AA0]-worker-SendThread(127.0.0.1:63073)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x1005bfa047d0001.
  2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x1005bfa047d0001, likely server has closed socket
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
  2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x1005bfa047d0001, likely server has closed socket
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
  2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
  2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
  2> 394521 WARN  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Watch limit violations: 
  2> Maximum concurrent create/delete watches above limit:
  2> 
  2> 	12	/solr/configs/conf
  2> 	7	/solr/aliases.json
  2> 	6	/solr/packages.json
  2> 	6	/solr/security.json
  2> 	6	/solr/collections/collection28/terms/shard2
  2> 	5	/solr/collections/collection28/terms/shard3
  2> 	5	/solr/collections/collection28/terms/shard1
  2> 
  2> Maximum concurrent data watches above limit:
  2> 
  2> 	80	/solr/collections/collection28/state.json
  2> 	7	/solr/clusterprops.json
  2> 	2	/solr/overseer_elect/election/72158723413180430-127.0.0.1:63088_solr-n_0000000002
  2> 
  2> Maximum concurrent children watches above limit:
  2> 
  2> 	45	/solr/live_nodes
  2> 	39	/solr/overseer/queue
  2> 	14	/solr/collections
  2> 	3	/solr/overseer/collection-queue-work
  2> 
  2> 394550 INFO  (SUITE-CustomHighlightComponentTest-seed#[C121695712E93AA0]-worker) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:63073
  2> 394550 INFO  (SUITE-CustomHighlightComponentTest-seed#[C121695712E93AA0]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:63073
  2> 394550 INFO  (SUITE-CustomHighlightComponentTest-seed#[C121695712E93AA0]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 63073
  2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/tmp/tests-tmp/solr.handler.component.CustomHighlightComponentTest_C121695712E93AA0-001
  2> Oct 09, 2020 5:49:24 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
  2> NOTE: test params are: codec=Lucene87, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=en-IN, timezone=PST8PDT
  2> NOTE: Mac OS X 10.14.6 x86_64/AdoptOpenJDK 12.0.2 (64-bit)/cpus=6,threads=1,free=106689336,total=380108800
  2> NOTE: All tests run in this JVM: [V2StandaloneTest, XsltUpdateRequestHandlerTest, TestHdfsBackupRestoreCore, CoreAdminRequestStatusTest, ShowFileRequestHandlerTest, DaemonStreamApiTest, ZookeeperReadAPITest, IndexSizeEstimatorTest, TestCollectionAPIs, TestPivotHelperCode, SpellCheckComponentTest, QueryElevationComponentTest, CustomHighlightComponentTest]


org.apache.solr.handler.TestSolrConfigHandlerCloud > test FAILED
    java.lang.AssertionError: {
      "responseHeader":{
        "status":500,
        "QTime":30131},
      "errorMessages":["4 out of 5 the property overlay to be of version 3 within 30 seconds! Failed cores: [http://127.0.0.1:63166/zahww/cq/collection1_shard2_replica_n2/, http://127.0.0.1:63194/zahww/cq/collection1_shard2_replica_n5/, http://127.0.0.1:63179/zahww/cq/collection1_shard1_replica_n1/, http://127.0.0.1:63215/zahww/cq/collection1_shard1_replica_n6/]\n"],
      "WARNING":"This response format is experimental.  It is likely to change in the future.",
      "error":{
        "metadata":[
          "error-class","org.apache.solr.common.SolrException",
          "root-error-class","org.apache.solr.common.SolrException"],
        "msg":"4 out of 5 the property overlay to be of version 3 within 30 seconds! Failed cores: [http://127.0.0.1:63166/zahww/cq/collection1_shard2_replica_n2/, http://127.0.0.1:63194/zahww/cq/collection1_shard2_replica_n5/, http://127.0.0.1:63179/zahww/cq/collection1_shard1_replica_n1/, http://127.0.0.1:63215/zahww/cq/collection1_shard1_replica_n6/]",
        "trace":"org.apache.solr.common.SolrException: 4 out of 5 the property overlay to be of version 3 within 30 seconds! Failed cores: [http://127.0.0.1:63166/zahww/cq/collection1_shard2_replica_n2/, http://127.0.0.1:63194/zahww/cq/collection1_shard2_replica_n5/, http://127.0.0.1:63179/zahww/cq/collection1_shard1_replica_n1/, http://127.0.0.1:63215/zahww/cq/collection1_shard1_replica_n6/]\n\tat org.apache.solr.handler.SolrConfigHandler.waitForAllReplicasState(SolrConfigHandler.java:827)\n\tat org.apache.solr.handler.SolrConfigHandler$Command.handleCommands(SolrConfigHandler.java:544)\n\tat org.apache.solr.handler.SolrConfigHandler$Command.handlePOST(SolrConfigHandler.java:376)\n\tat org.apache.solr.handler.SolrConfigHandler.handleRequestBody(SolrConfigHandler.java:138)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:212)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:2610)\n\tat org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:795)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:568)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:451)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:365)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doF

[...truncated too long message...]

sass-lang.com/blog/posts/7828841

Successfully installed sass-3.7.4
Successfully installed public_suffix-4.0.6
Successfully installed temple-0.8.2
Successfully installed asciidoctor-2.0.10
Successfully installed safe_yaml-1.0.5
Successfully installed rouge-1.11.1
Successfully installed pathutil-0.16.2
Successfully installed mercenary-0.3.6
Successfully installed liquid-4.0.3
Successfully installed kramdown-1.17.0
Successfully installed jekyll-watch-1.5.1
Successfully installed jekyll-sass-converter-1.5.2
Successfully installed colorator-1.1.0
Successfully installed addressable-2.7.0
Successfully installed concurrent-ruby-1.0.5-java
Successfully installed tilt-2.0.10
Successfully installed slim-4.0.1
Successfully installed jekyll-3.5.2
Successfully installed jekyll-asciidoc-3.0.0
25 gems installed

> Task :solr:solr-ref-guide:buildSiteJekyll
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/Users/jenkins/.gradle/caches/modules-2/files-2.1/org.jruby/jruby-complete/9.2.9.0/8e44ce7a1417966d89957bd766a148601e28828b/jruby-complete-9.2.9.0.jar) to method sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Configuration file: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/solr-ref-guide/build/content/_config.yml
            Source: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/solr-ref-guide/build/content
       Destination: ../html-site
 Incremental build: disabled. Enable with --incremental
      Generating... 
                    done in 33.419 seconds.
 Auto-regeneration: disabled. Use --watch to enable.

> Task :solr:solr-ref-guide:buildSite
Processed 2450 links (1766 relative) to 3536 anchors in 245 files

> Task :solr:solr-ref-guide:check
> Task :solr:solrj:check
> Task :solr:test-framework:check
> Task :solr:webapp:check
> Task :lucene:analysis:common:check
> Task :lucene:analysis:icu:check
> Task :lucene:analysis:kuromoji:check
> Task :lucene:analysis:morfologik:check
> Task :lucene:analysis:nori:check
> Task :lucene:analysis:opennlp:check
> Task :lucene:analysis:phonetic:check
> Task :lucene:analysis:smartcn:check
> Task :lucene:analysis:stempel:check
> Task :solr:contrib:analysis-extras:check
> Task :solr:contrib:analytics:check
> Task :solr:contrib:clustering:check
> Task :solr:contrib:extraction:check
> Task :solr:contrib:jaegertracer-configurator:check
> Task :solr:contrib:langid:check
> Task :solr:contrib:ltr:check
> Task :solr:contrib:prometheus-exporter:check
> Task :solr:docker:package:check
The slowest tests (exceeding 500 ms) during this run:
  404.32s TestDistributedSearch.test (:solr:core)
  233.34s TestDistributedStatsComponentCardinality.test (:solr:core)
  155.84s HdfsDirectoryTest.testRandomAccessWrites (:solr:core)
  140.39s TestSolrConfigHandlerCloud.test (:solr:core)
  132.35s TestSolrCachePerf.testGetPutCompute (:solr:core)
  130.30s HttpPartitionWithTlogReplicasTest.test (:solr:core)
  111.50s TestCloudSchemaless.test (:solr:core)
  106.81s ChaosMonkeySafeLeaderWithPullReplicasTest.test (:solr:core)
  93.28s TestSolrConfigHandlerConcurrent.test (:solr:core)
  86.88s DistributedIntervalFacetingTest.test (:solr:core)

ERROR: The following test(s) have failed:
  - org.apache.solr.handler.TestSystemCollAutoCreate.testAutoCreate (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.TestSystemCollAutoCreate.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.handler.TestSystemCollAutoCreate" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UseParallelGC" -Ptests.seed=C121695712E93AA0 -Ptests.badapples=false -Ptests.file.encoding=UTF-8

  - org.apache.solr.handler.component.CustomHighlightComponentTest.test (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.CustomHighlightComponentTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.handler.component.CustomHighlightComponentTest" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UseParallelGC" -Ptests.seed=C121695712E93AA0 -Ptests.badapples=false -Ptests.file.encoding=UTF-8

  - org.apache.solr.handler.TestSolrConfigHandlerCloud.test (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.TestSolrConfigHandlerCloud.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.handler.TestSolrConfigHandlerCloud" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UseParallelGC" -Ptests.seed=C121695712E93AA0 -Ptests.badapples=false -Ptests.file.encoding=UTF-8


Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 1h 29m 11s
842 actionable tasks: 842 executed
Build step 'Invoke Gradle script' changed build result to SUCCESS
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2875)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2754)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2735)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3122)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to MacOSX VBOX
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:1001)
		at hudson.FilePath.act(FilePath.java:1070)
		at hudson.FilePath.act(FilePath.java:1059)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2733)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:270)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:753)
		at hudson.model.Build$BuildExecution.post2(Build.java:177)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:698)
		at hudson.model.Run.execute(Run.java:1919)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3124)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:375)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1072)
	at hudson.FilePath.act(FilePath.java:1059)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2733)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:270)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:753)
	at hudson.model.Build$BuildExecution.post2(Build.java:177)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:698)
	at hudson.model.Run.execute(Run.java:1919)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/*_pid*.log". Configuration error?
[Java] Sleeping for 5 seconds due to JENKINS-32191...
[Java] Parsing console log (workspace: '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX')
[Java] Successfully parsed console log
[Java] -> found 0 issues (skipped 0 duplicates)
[Java] Skipping post processing
[Java] No filter has been set, publishing all 0 issues
[Java] Using reference build 'Lucene-Solr-master-MacOSX #6609' to compute new, fixed, and outstanding issues
[Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
[Java] No quality gates have been set - skipping
[Java] Health report is disabled - skipping
[Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)
[Java] Attaching ResultAction with ID 'java' to run 'Lucene-Solr-master-MacOSX #6610'.
[Checks API] No suitable checks publisher found.
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
[Checks API] No suitable checks publisher found.
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-13.0.2) - Build # 6614 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6614/
Java: 64bit/jdk-13.0.2 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

1 tests failed.
FAILED:  org.apache.solr.cloud.PeerSyncReplicationTest.test

Error Message:
java.lang.AssertionError: expected:<154> but was:<152>

Stack Trace:
java.lang.AssertionError: expected:<154> but was:<152>
	at __randomizedtesting.SeedInfo.seed([B21FF84D43D29335:3A4BC797ED2EFECD]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:834)
	at org.junit.Assert.assertEquals(Assert.java:645)
	at org.junit.Assert.assertEquals(Assert.java:631)
	at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:154)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1090)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1061)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:830)




Build Log:
[...truncated 4852 lines...]
ERROR: The following test(s) have failed:
  - org.apache.solr.cloud.PeerSyncReplicationTest.test (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.PeerSyncReplicationTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.PeerSyncReplicationTest" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:+UseCompressedOops -XX:+UseConcMarkSweepGC" -Ptests.seed=B21FF84D43D29335 -Ptests.badapples=false -Ptests.file.encoding=UTF-8


Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 1h 19m 2s
849 actionable tasks: 849 executed
Build step 'Invoke Gradle script' changed build result to SUCCESS
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2875)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2754)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2735)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3122)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to MacOSX VBOX
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:1001)
		at hudson.FilePath.act(FilePath.java:1070)
		at hudson.FilePath.act(FilePath.java:1059)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2733)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:270)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:753)
		at hudson.model.Build$BuildExecution.post2(Build.java:177)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:698)
		at hudson.model.Run.execute(Run.java:1919)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3124)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:375)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1072)
	at hudson.FilePath.act(FilePath.java:1059)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2733)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:270)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:753)
	at hudson.model.Build$BuildExecution.post2(Build.java:177)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:698)
	at hudson.model.Run.execute(Run.java:1919)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/*_pid*.log". Configuration error?
[Java] Sleeping for 5 seconds due to JENKINS-32191...
[Java] Parsing console log (workspace: '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX')
[Java] Successfully parsed console log
[Java] -> found 0 issues (skipped 0 duplicates)
[Java] Skipping post processing
[Java] No filter has been set, publishing all 0 issues
[Java] Using reference build 'Lucene-Solr-master-MacOSX #6613' to compute new, fixed, and outstanding issues
[Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
[Java] No quality gates have been set - skipping
[Java] Health report is disabled - skipping
[Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)
[Java] Attaching ResultAction with ID 'java' to run 'Lucene-Solr-master-MacOSX #6614'.
[Checks API] No suitable checks publisher found.
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
[Checks API] No suitable checks publisher found.
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-15) - Build # 6613 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6613/
Java: 64bit/jdk-15 -XX:-UseCompressedOops -XX:+UseG1GC

3 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionWithTlogReplicasTest.test

Error Message:
java.lang.AssertionError: Doc with id=2 not found in http://127.0.0.1:54205/c8n_1x2_leader_session_loss due to: Path not found: /id; rsp={doc=null}

Stack Trace:
java.lang.AssertionError: Doc with id=2 not found in http://127.0.0.1:54205/c8n_1x2_leader_session_loss due to: Path not found: /id; rsp={doc=null}
	at __randomizedtesting.SeedInfo.seed([417611789BCE826C:C9222EA23532EF94]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.apache.solr.cloud.HttpPartitionTest.assertDocExists(HttpPartitionTest.java:566)
	at org.apache.solr.cloud.HttpPartitionTest.assertDocsExistInAllReplicas(HttpPartitionTest.java:511)
	at org.apache.solr.cloud.HttpPartitionTest.testLeaderZkSessionLoss(HttpPartitionTest.java:462)
	at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:148)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1090)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1061)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.handler.component.CustomHighlightComponentTest.test

Error Message:
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:64806/solr/collection49: error processing commands

Stack Trace:
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:64806/solr/collection49: error processing commands
	at __randomizedtesting.SeedInfo.seed([417611789BCE826C:C9222EA23532EF94]:0)
	at org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException.create(BaseHttpSolrClient.java:67)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:664)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:269)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
	at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:390)
	at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:360)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1170)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:931)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:865)
	at org.apache.solr.handler.component.CustomHighlightComponentTest.test(CustomHighlightComponentTest.java:152)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.cloud.HttpPartitionOnCommitTest.test

Error Message:
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:61011: Underlying core creation failed while creating collection: c8n_2x2_commits

Stack Trace:
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:61011: Underlying core creation failed while creating collection: c8n_2x2_commits
	at __randomizedtesting.SeedInfo.seed([417611789BCE826C:C9222EA23532EF94]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:698)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:269)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1290)
	at org.apache.solr.cloud.BasicDistributedZkTest.createCollection(BasicDistributedZkTest.java:809)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1755)
	at org.apache.solr.cloud.HttpPartitionOnCommitTest.multiShardTest(HttpPartitionOnCommitTest.java:76)
	at org.apache.solr.cloud.HttpPartitionOnCommitTest.test(HttpPartitionOnCommitTest.java:67)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1090)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1061)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)




Build Log:
[...truncated 7750 lines...]
  2> 258313 ERROR (qtp1805895934-1136) [n:127.0.0.1:64806_solr c:collection49 s:shard2 r:core_node7 x:collection49_shard2_replica_n4 ] o.a.s.h.SolrConfigHandler ERROR:[{
  2>     "add-searchcomponent":{
  2>       "name":"customhighlight-727794035",
  2>       "class":"org.apache.solr.handler.component.CustomHighlightComponentTest$CustomHighlightComponent"},
  2>     "errorMessages":[" 'customhighlight-727794035' already exists . Do an 'update-searchcomponent' , if you want to change it "]}]
  2> 258314 INFO  (qtp1805895934-1136) [n:127.0.0.1:64806_solr c:collection49 s:shard2 r:core_node7 x:collection49_shard2_replica_n4 ] o.a.s.c.S.Request [collection49_shard2_replica_n4]  webapp=/solr path=/config params={wt=javabin&version=2} status=400 QTime=56
  2> 258318 INFO  (TEST-CustomHighlightComponentTest.test-seed#[417611789BCE826C]) [     ] o.a.s.c.s.i.BaseCloudSolrClient request was not communication error it seems
  2> 258318 INFO  (TEST-CustomHighlightComponentTest.test-seed#[417611789BCE826C]) [     ] o.a.s.c.s.i.BaseCloudSolrClient Request to collection [collection49] failed due to (400) org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:64806/solr/collection49: error processing commands, retry=0 maxRetries=5 commError=false errorCode=400 
  2> 258320 INFO  (TEST-CustomHighlightComponentTest.test-seed#[417611789BCE826C]) [     ] o.a.s.SolrTestCaseJ4 ###Ending test
   >     org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException: Error from server at http://127.0.0.1:64806/solr/collection49: error processing commands
   >         at __randomizedtesting.SeedInfo.seed([417611789BCE826C:C9222EA23532EF94]:0)
   >         at org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteExecutionException.create(BaseHttpSolrClient.java:67)
   >         at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:664)
   >         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:269)
   >         at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
   >         at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:390)
   >         at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:360)
   >         at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1170)
   >         at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:931)
   >         at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:865)
   >         at org.apache.solr.handler.component.CustomHighlightComponentTest.test(CustomHighlightComponentTest.java:152)
   >         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   >         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
   >         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   >         at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
   >         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   >         at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   >         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   >         at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   >         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   >         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   >         at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   >         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   >         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   >         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   >         at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   >         at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
   >         at java.base/java.lang.Thread.run(Thread.java:832)
  2> NOTE: reproduce with: gradlew test --tests CustomHighlightComponentTest.test -Dtests.seed=417611789BCE826C -Dtests.slow=true -Dtests.locale=pt-CH -Dtests.timezone=Asia/Tehran -Dtests.asserts=true -Dtests.file.encoding=UTF-8
  2> 258537 INFO  (qtp1192337113-1122) [n:127.0.0.1:64803_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.collection49.shard2.replica_n6:INDEX.sizeInBytes&key=solr.core.collection49.shard2.replica_n6:QUERY./select.requests&key=solr.core.collection49.shard2.replica_n6:UPDATE./update.requests} status=0 QTime=2
  2> 258667 INFO  (qtp1192337113-1132) [n:127.0.0.1:64803_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.jvm:os.processCpuLoad&key=solr.node:CONTAINER.fs.coreRoot.usableSpace&key=solr.jvm:os.systemLoadAverage&key=solr.jvm:memory.heap.used} status=0 QTime=12
  2> 258700 INFO  (jetty-closer-1917-thread-3) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2003537170
  2> 258700 INFO  (jetty-closer-1917-thread-3) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:64803_solr
  2> 258706 INFO  (jetty-closer-1917-thread-2) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=992917280
  2> 258706 INFO  (jetty-closer-1917-thread-2) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:64804_solr
  2> 258717 INFO  (jetty-closer-1917-thread-4) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1955722570
  2> 258718 INFO  (jetty-closer-1917-thread-4) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:64802_solr
  2> 258753 INFO  (jetty-closer-1917-thread-5) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=505845731
  2> 258753 INFO  (jetty-closer-1917-thread-5) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:64800_solr
  2> 258758 INFO  (jetty-closer-1917-thread-6) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=961772091
  2> 258758 INFO  (jetty-closer-1917-thread-6) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:64805_solr
  2> 258878 INFO  (jetty-closer-1917-thread-3) [     ] o.a.s.c.ZkController Publish this node as DOWN...
  2> 258878 INFO  (jetty-closer-1917-thread-3) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:64803_solr as DOWN
  2> 258880 INFO  (jetty-closer-1917-thread-4) [     ] o.a.s.c.ZkController Publish this node as DOWN...
  2> 258880 INFO  (jetty-closer-1917-thread-4) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:64802_solr as DOWN
  2> 258897 INFO  (jetty-closer-1917-thread-5) [     ] o.a.s.c.ZkController Publish this node as DOWN...
  2> 258897 INFO  (jetty-closer-1917-thread-5) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:64800_solr as DOWN
  2> 258898 INFO  (jetty-closer-1917-thread-6) [     ] o.a.s.c.ZkController Publish this node as DOWN...
  2> 258898 INFO  (jetty-closer-1917-thread-6) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:64805_solr as DOWN
  2> 258921 INFO  (jetty-closer-1917-thread-2) [     ] o.a.s.c.ZkController Publish this node as DOWN...
  2> 258921 INFO  (jetty-closer-1917-thread-2) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:64804_solr as DOWN
  2> 258923 ERROR (qtp1805895934-1136) [n:127.0.0.1:64806_solr     ] o.a.s.s.SolrDispatchFilter Error processing the request. CoreContainer is either not initialized or shutting down.
  2> 258924 WARN  (qtp1805895934-1136) [n:127.0.0.1:64806_solr     ] o.e.j.s.HttpChannel /solr/admin/metrics
  2>           => javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
  2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:385)
  2> javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
  2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:385) ~[main/:?]
  2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:365) ~[main/:?]
  2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604) ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166) ~[main/:?]
  2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604) ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322) ~[jetty-rewrite-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.Server.handle(Server.java:500) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) [jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) [jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) [jetty-io-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) [jetty-io-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) [jetty-util-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) [jetty-util-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at java.lang.Thread.run(Thread.java:832) [?:?]
  2> 258996 INFO  (zkCallback-644-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (1)
  2> 259012 INFO  (zkCallback-656-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (1)
  2> 259021 INFO  (zkCallback-668-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (1)
  2> 259022 INFO  (zkCallback-630-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (1)
  2> 259023 INFO  (zkCallback-620-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (1)
  2> 259025 INFO  (zkCallback-683-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (1)
  2> 259310 INFO  (coreCloseExecutor-1936-thread-1) [n:127.0.0.1:64805_solr     ] o.a.s.c.SolrCore [collection49_shard3_replica_n10]  CLOSING SolrCore org.apache.solr.core.SolrCore@7362478a
  2> 259310 INFO  (coreCloseExecutor-1936-thread-1) [n:127.0.0.1:64805_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection49.shard3.replica_n10 tag=SolrCore@7362478a
  2> 259310 INFO  (coreCloseExecutor-1936-thread-1) [n:127.0.0.1:64805_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3e628f87: rootName = solr_64805, domain = solr.core.collection49.shard3.replica_n10, service url = null, agent id = null] for registry solr.core.collection49.shard3.replica_n10/com.codahale.metrics.MetricRegistry@b0afdd0
  2> 259323 INFO  (coreCloseExecutor-1933-thread-1) [n:127.0.0.1:64803_solr     ] o.a.s.c.SolrCore [collection49_shard2_replica_n6]  CLOSING SolrCore org.apache.solr.core.SolrCore@6df360c1
  2> 259323 INFO  (coreCloseExecutor-1937-thread-1) [n:127.0.0.1:64800_solr     ] o.a.s.c.SolrCore [collection49_shard1_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@3b4a0dd1
  2> 259323 INFO  (coreCloseExecutor-1933-thread-1) [n:127.0.0.1:64803_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection49.shard2.replica_n6 tag=SolrCore@6df360c1
  2> 259323 INFO  (coreCloseExecutor-1937-thread-1) [n:127.0.0.1:64800_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection49.shard1.replica_n2 tag=SolrCore@3b4a0dd1
  2> 259323 INFO  (coreCloseExecutor-1933-thread-1) [n:127.0.0.1:64803_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7462d9: rootName = solr_64803, domain = solr.core.collection49.shard2.replica_n6, service url = null, agent id = null] for registry solr.core.collection49.shard2.replica_n6/com.codahale.metrics.MetricRegistry@4259b5a5
  2> 259323 INFO  (coreCloseExecutor-1937-thread-1) [n:127.0.0.1:64800_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@45fb939d: rootName = solr_64800, domain = solr.core.collection49.shard1.replica_n2, service url = null, agent id = null] for registry solr.core.collection49.shard1.replica_n2/com.codahale.metrics.MetricRegistry@18cb2c01
  2> 259324 INFO  (coreCloseExecutor-1931-thread-1) [n:127.0.0.1:64804_solr     ] o.a.s.c.SolrCore [collection49_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@18f31db6
  2> 259324 INFO  (coreCloseExecutor-1931-thread-1) [n:127.0.0.1:64804_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection49.shard1.replica_n1 tag=SolrCore@18f31db6
  2> 259324 INFO  (coreCloseExecutor-1931-thread-1) [n:127.0.0.1:64804_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@230ff0c0: rootName = solr_64804, domain = solr.core.collection49.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection49.shard1.replica_n1/com.codahale.metrics.MetricRegistry@776124a0
  2> 259376 WARN  (MetricsHistoryHandler-691-thread-1) [     ] o.a.s.c.s.i.SolrClientNodeStateProvider could not get tags from node 127.0.0.1:64806_solr
  2>           => org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:64806/solr: Expected mime type application/octet-stream but got text/html. <html>
  2> <head>
  2> org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:64806/solr: Expected mime type application/octet-stream but got text/html. <html>
  2> <head>
  2> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
  2> <title>Error 404 javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</title>
  2> </head>
  2> <body><h2>HTTP ERROR 404 javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</h2>
  2> <table>
  2> <tr><th>URI:</th><td>/solr/admin/metrics</td></tr>
  2> <tr><th>STATUS:</th><td>404</td></tr>
  2> <tr><th>MESSAGE:</th><td>javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</td></tr>
  2> <tr><th>SERVLET:</th><td>org.apache.solr.client.solrj.embedded.JettySolrRunner$Servlet404-5148f98c</td></tr>
  2> <tr><th>CAUSED BY:</th><td>javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</td></tr>
  2> </table>
  2> <h3>Caused by:</h3><pre>javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
  2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:385)
  2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:365)
  2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
  2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
  2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
  2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
  2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
  2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
  2> 	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
  2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
  2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
  2> 	at org.eclipse.jetty.server.Server.handle(Server.java:500)
  2> 	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
  2> 	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
  2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
  2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
  2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
  2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
  2> 	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
  2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
  2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
  2> 	at java.base/java.lang.Thread.run(Thread.java:832)
  2> </pre>
  2> <hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.27.v20200227</a><hr/>
  2> 
  2> </body>
  2> </html>
  2> 
  2> 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:652) ~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:269) ~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251) ~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1290) ~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$ClientSnitchCtx.invoke(SolrClientNodeStateProvider.java:381) ~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$ClientSnitchCtx.invokeWithRetry(SolrClientNodeStateProvider.java:344) ~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.fetchReplicaMetrics(SolrClientNodeStateProvider.java:188) [solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.fetchReplicaMetrics(SolrClientNodeStateProvider.java:178) [solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.getReplicaInfo(SolrClientNodeStateProvider.java:163) [solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.handler.admin.MetricsHistoryHandler.collectGlobalMetrics(MetricsHistoryHandler.java:493) [main/:?]
  2> 	at org.apache.solr.handler.admin.MetricsHistoryHandler.collectMetrics(MetricsHistoryHandler.java:381) [main/:?]
  2> 	at org.apache.solr.handler.admin.MetricsHistoryHandler.lambda$new$0(MetricsHistoryHandler.java:236) [main/:?]
  2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
  2> 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
  2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
  2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
  2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
  2> 	at java.lang.Thread.run(Thread.java:832) [?:?]
  2> 259381 ERROR (qtp1805895934-1136) [n:127.0.0.1:64806_solr     ] o.a.s.s.SolrDispatchFilter Error processing the request. CoreContainer is either not initialized or shutting down.
  2> 259382 WARN  (qtp1805895934-1136) [n:127.0.0.1:64806_solr     ] o.e.j.s.HttpChannel /solr/admin/metrics
  2>           => javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
  2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:385)
  2> javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
  2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:385) ~[main/:?]
  2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:365) ~[main/:?]
  2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604) ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166) ~[main/:?]
  2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604) ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545) ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485) ~[jetty-servlet-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322) ~[jetty-rewrite-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.Server.handle(Server.java:500) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547) ~[jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) [jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) [jetty-server-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103) [jetty-io-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) [jetty-io-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) [jetty-util-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) [jetty-util-9.4.27.v20200227.jar:9.4.27.v20200227]
  2> 	at java.lang.Thread.run(Thread.java:832) [?:?]
  2> 259408 WARN  (MetricsHistoryHandler-691-thread-1) [     ] o.a.s.c.s.i.SolrClientNodeStateProvider could not get tags from node 127.0.0.1:64806_solr
  2>           => org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:64806/solr: Expected mime type application/octet-stream but got text/html. <html>
  2> <head>
  2> org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:64806/solr: Expected mime type application/octet-stream but got text/html. <html>
  2> <head>
  2> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
  2> <title>Error 404 javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</title>
  2> </head>
  2> <body><h2>HTTP ERROR 404 javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</h2>
  2> <table>
  2> <tr><th>URI:</th><td>/solr/admin/metrics</td></tr>
  2> <tr><th>STATUS:</th><td>404</td></tr>
  2> <tr><th>MESSAGE:</th><td>javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</td></tr>
  2> <tr><th>SERVLET:</th><td>org.apache.solr.client.solrj.embedded.JettySolrRunner$Servlet404-5148f98c</td></tr>
  2> <tr><th>CAUSED BY:</th><td>javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.</td></tr>
  2> </table>
  2> <h3>Caused by:</h3><pre>javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down.
  2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:385)
  2> 	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:365)
  2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:166)
  2> 	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1604)
  2> 	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  2> 	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610)
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
  2> 	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1300)
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
  2> 	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:485)
  2> 	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580)
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
  2> 	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1215)
  2> 	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
  2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
  2> 	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:322)
  2> 	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:717)
  2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
  2> 	at org.eclipse.jetty.server.Server.handle(Server.java:500)
  2> 	at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
  2> 	at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
  2> 	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
  2> 	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
  2> 	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
  2> 	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
  2> 	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
  2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806)
  2> 	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938)
  2> 	at java.base/java.lang.Thread.run(Thread.java:832)
  2> </pre>
  2> <hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.27.v20200227</a><hr/>
  2> 
  2> </body>
  2> </html>
  2> 
  2> 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:652) ~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:269) ~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251) ~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1290) ~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$ClientSnitchCtx.invoke(SolrClientNodeStateProvider.java:381) ~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$ClientSnitchCtx.invokeWithRetry(SolrClientNodeStateProvider.java:344) ~[solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.fetchReplicaMetrics(SolrClientNodeStateProvider.java:188) [solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider$MetricsFetchingSnitch.getRemoteInfo(SolrClientNodeStateProvider.java:236) [solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.common.cloud.rule.ImplicitSnitch.getTags(ImplicitSnitch.java:75) [solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.fetchTagValues(SolrClientNodeStateProvider.java:131) [solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.client.solrj.impl.SolrClientNodeStateProvider.getNodeValues(SolrClientNodeStateProvider.java:123) [solr-solrj-9.0.0-SNAPSHOT.jar:9.0.0-SNAPSHOT 80df6a3d62cfacf6144f17a04447cf7707de6de1 [snapshot build, details omitted]]
  2> 	at org.apache.solr.handler.admin.MetricsHistoryHandler.collectGlobalMetrics(MetricsHistoryHandler.java:510) [main/:?]
  2> 	at org.apache.solr.handler.admin.MetricsHistoryHandler.collectMetrics(MetricsHistoryHandler.java:381) [main/:?]
  2> 	at org.apache.solr.handler.admin.MetricsHistoryHandler.lambda$new$0(MetricsHistoryHandler.java:236) [main/:?]
  2> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
  2> 	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
  2> 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?]
  2> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
  2> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
  2> 	at java.lang.Thread.run(Thread.java:832) [?:?]
  2> 259555 INFO  (jetty-closer-1917-thread-1) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2024214599
  2> 259555 INFO  (jetty-closer-1917-thread-1) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:64806_solr
  2> 259577 INFO  (jetty-closer-1917-thread-1) [     ] o.a.s.c.ZkController Publish this node as DOWN...
  2> 259577 INFO  (jetty-closer-1917-thread-1) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:64806_solr as DOWN
  2> 259582 INFO  (zkCallback-644-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
  2> 259599 INFO  (zkCallback-620-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
  2> 259602 INFO  (zkCallback-656-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
  2> 259603 INFO  (zkCallback-656-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259603 INFO  (zkCallback-630-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
  2> 259605 INFO  (zkCallback-644-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259606 INFO  (zkCallback-656-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259607 INFO  (zkCallback-668-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
  2> 259607 INFO  (zkCallback-668-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259607 INFO  (zkCallback-683-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [1])
  2> 259608 INFO  (zkCallback-630-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259608 INFO  (zkCallback-630-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259608 INFO  (zkCallback-620-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259608 INFO  (zkCallback-668-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259608 INFO  (zkCallback-644-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259609 INFO  (zkCallback-683-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [1])
  2> 259611 INFO  (zkCallback-620-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259612 INFO  (zkCallback-683-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
  2> 259614 INFO  (zkCallback-620-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259686 INFO  (coreCloseExecutor-1942-thread-1) [n:127.0.0.1:64806_solr     ] o.a.s.c.SolrCore [collection49_shard2_replica_n4]  CLOSING SolrCore org.apache.solr.core.SolrCore@5757aad8
  2> 259686 INFO  (coreCloseExecutor-1942-thread-1) [n:127.0.0.1:64806_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection49.shard2.replica_n4 tag=SolrCore@5757aad8
  2> 259686 INFO  (coreCloseExecutor-1942-thread-1) [n:127.0.0.1:64806_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7e961da0: rootName = solr_64806, domain = solr.core.collection49.shard2.replica_n4, service url = null, agent id = null] for registry solr.core.collection49.shard2.replica_n4/com.codahale.metrics.MetricRegistry@50f58c57
  2> 259763 INFO  (searcherExecutor-787-thread-1-processing-n:127.0.0.1:64802_solr) [n:127.0.0.1:64802_solr     ] o.a.s.c.SolrCore [collection49_shard3_replica_n8]  Registered new searcher autowarm time: 0 ms
  2> 259764 INFO  (qtp730558419-1114) [n:127.0.0.1:64802_solr c:collection49 s:shard3 r:core_node11 x:collection49_shard3_replica_n8 ] o.a.s.h.SolrConfigHandler I already have the expected version 0 of config
  2> 259764 INFO  (qtp730558419-1114) [n:127.0.0.1:64802_solr c:collection49 s:shard3 r:core_node11 x:collection49_shard3_replica_n8 ] o.a.s.h.SolrConfigHandler isStale false , resourceloader org.apache.solr.cloud.ZkSolrResourceLoader
  2> 259764 INFO  (qtp730558419-1114) [n:127.0.0.1:64802_solr c:collection49 s:shard3 r:core_node11 x:collection49_shard3_replica_n8 ] o.a.s.c.S.Request [collection49_shard3_replica_n8]  webapp=/solr path=/config/znodeVersion params={overlay=0&wt=javabin&version=2} status=0 QTime=3495
  2> 259764 INFO  (qtp730558419-1114) [n:127.0.0.1:64802_solr     ] o.a.s.c.SolrCore [collection49_shard3_replica_n8]  CLOSING SolrCore org.apache.solr.core.SolrCore@1a0025ad
  2> 259928 INFO  (zkCallback-656-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259928 INFO  (zkCallback-668-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259938 INFO  (zkCallback-668-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259938 INFO  (zkCallback-620-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259969 INFO  (zkCallback-683-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259969 INFO  (zkCallback-656-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259970 INFO  (zkCallback-683-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259979 INFO  (zkCallback-630-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259983 INFO  (zkCallback-630-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 259997 INFO  (zkCallback-644-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 260012 INFO  (zkCallback-644-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 260030 INFO  (zkCallback-620-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 260044 INFO  (zkCallback-620-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection49/state.json] for collection [collection49] has occurred - updating... (live nodes size: [0])
  2> 261973 INFO  (Thread-349) [n:127.0.0.1:64800_solr c:collection49 s:shard1 r:core_node5 x:collection49_shard1_replica_n2 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64800.solr.core.collection49.shard1.replica_n2' (registry 'solr.core.collection49.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42967c7a
  2> 262682 INFO  (Thread-351) [n:127.0.0.1:64804_solr c:collection49 s:shard1 r:core_node3 x:collection49_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64804.solr.core.collection49.shard1.replica_n1' (registry 'solr.core.collection49.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42967c7a
  2> 264394 INFO  (Thread-353) [n:127.0.0.1:64802_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection49.shard3.leader tag=SolrCore@f461efb
  2> 264394 INFO  (jetty-closer-1917-thread-4) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
  2> 264395 INFO  (jetty-closer-1917-thread-4) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@14bbb0f1: rootName = solr_64802, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@18648f48
  2> 264994 INFO  (coreCloseExecutor-1936-thread-1) [n:127.0.0.1:64805_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection49.shard3.leader tag=SolrCore@7362478a
  2> 265013 INFO  (coreCloseExecutor-1933-thread-1) [n:127.0.0.1:64803_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection49.shard2.leader tag=SolrCore@6df360c1
  2> 265063 INFO  (coreCloseExecutor-1931-thread-1) [n:127.0.0.1:64804_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection49.shard1.leader tag=SolrCore@18f31db6
  2> 265064 INFO  (Thread-351) [n:127.0.0.1:64804_solr     ] o.a.s.c.SolrCore [[collection49_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/tmp/tests-tmp/solr.handler.component.CustomHighlightComponentTest_417611789BCE826C-001/tempDir-001/node6/collection49_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/tmp/tests-tmp/solr.handler.component.CustomHighlightComponentTest_417611789BCE826C-001/tempDir-001/node6/collection49_shard1_replica_n1/data/]
  2> 265065 INFO  (coreCloseExecutor-1937-thread-1) [n:127.0.0.1:64800_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection49.shard1.leader tag=SolrCore@3b4a0dd1
  2> 265065 INFO  (Thread-349) [n:127.0.0.1:64800_solr     ] o.a.s.c.SolrCore [[collection49_shard1_replica_n2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/tmp/tests-tmp/solr.handler.component.CustomHighlightComponentTest_417611789BCE826C-001/tempDir-001/node3/collection49_shard1_replica_n2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/tmp/tests-tmp/solr.handler.component.CustomHighlightComponentTest_417611789BCE826C-001/tempDir-001/node3/collection49_shard1_replica_n2/data/]
  2> 265068 INFO  (coreCloseExecutor-1942-thread-1) [n:127.0.0.1:64806_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection49.shard2.leader tag=SolrCore@5757aad8
  2> 267401 INFO  (Thread-350) [n:127.0.0.1:64806_solr c:collection49 s:shard2 r:core_node7 x:collection49_shard2_replica_n4 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64806.solr.core.collection49.shard2.replica_n4' (registry 'solr.core.collection49.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42967c7a
  2> 267406 INFO  (jetty-closer-1917-thread-5) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
  2> 267405 INFO  (Thread-350) [n:127.0.0.1:64806_solr     ] o.a.s.c.SolrCore [[collection49_shard2_replica_n4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/tmp/tests-tmp/solr.handler.component.CustomHighlightComponentTest_417611789BCE826C-001/tempDir-001/node1/collection49_shard2_replica_n4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/tmp/tests-tmp/solr.handler.component.CustomHighlightComponentTest_417611789BCE826C-001/tempDir-001/node1/collection49_shard2_replica_n4/data/]
  2> 267406 INFO  (jetty-closer-1917-thread-5) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@64a7402f: rootName = solr_64800, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@4fc77aa7
  2> 267476 INFO  (jetty-closer-1917-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
  2> 267476 INFO  (jetty-closer-1917-thread-2) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@247b9ebd: rootName = solr_64804, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@180c3019
  2> 267625 INFO  (Thread-349) [n:127.0.0.1:64800_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
  2> 267629 INFO  (Thread-349) [n:127.0.0.1:64800_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
  2> 267663 INFO  (Thread-351) [n:127.0.0.1:64804_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
  2> 267663 INFO  (Thread-351) [n:127.0.0.1:64804_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
  2> 268058 INFO  (Thread-349) [n:127.0.0.1:64800_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
  2> 268062 INFO  (Thread-349) [n:127.0.0.1:64800_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
  2> 268062 INFO  (Thread-349) [n:127.0.0.1:64800_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
  2> 268063 INFO  (Thread-349) [n:127.0.0.1:64800_solr     ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1680150470306299904
  2> 268079 INFO  (Thread-351) [n:127.0.0.1:64804_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
  2> 268085 INFO  (Thread-349) [n:127.0.0.1:64800_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
  2> 268088 INFO  (Thread-351) [n:127.0.0.1:64804_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
  2> 268088 INFO  (Thread-351) [n:127.0.0.1:64804_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
  2> 268089 INFO  (Thread-351) [n:127.0.0.1:64804_solr     ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1680150470333562880
  2> 268154 INFO  (Thread-351) [n:127.0.0.1:64804_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
  2> 268172 INFO  (Thread-352) [n:127.0.0.1:64805_solr c:collection49 s:shard3 r:core_node12 x:collection49_shard3_replica_n10 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64805.solr.core.collection49.shard3.replica_n10' (registry 'solr.core.collection49.shard3.replica_n10') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42967c7a
  2> 268172 INFO  (Thread-352) [n:127.0.0.1:64805_solr     ] o.a.s.c.SolrCore [[collection49_shard3_replica_n10] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/tmp/tests-tmp/solr.handler.component.CustomHighlightComponentTest_417611789BCE826C-001/tempDir-001/node4/collection49_shard3_replica_n10], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/tmp/tests-tmp/solr.handler.component.CustomHighlightComponentTest_417611789BCE826C-001/tempDir-001/node4/collection49_shard3_replica_n10/data/]
  2> 268290 INFO  (Thread-348) [n:127.0.0.1:64803_solr c:collection49 s:shard2 r:core_node9 x:collection49_shard2_replica_n6 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_64803.solr.core.collection49.shard2.replica_n6' (registry 'solr.core.collection49.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@42967c7a
  2> 268290 INFO  (Thread-348) [n:127.0.0.1:64803_solr     ] o.a.s.c.SolrCore [[collection49_shard2_replica_n6] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/tmp/tests-tmp/solr.handler.component.CustomHighlightComponentTest_417611789BCE826C-001/tempDir-001/node2/collection49_shard2_replica_n6], dataDir=[/Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/tmp/tests-tmp/solr.handler.component.CustomHighlightComponentTest_417611789BCE826C-001/tempDir-001/node2/collection49_shard2_replica_n6/data/]
  2> 268370 INFO  (searcherExecutor-801-thread-1-processing-n:127.0.0.1:64800_solr) [n:127.0.0.1:64800_solr     ] o.a.s.c.SolrCore [collection49_shard1_replica_n2]  Registered new searcher autowarm time: 0 ms
  2> 268371 INFO  (searcherExecutor-798-thread-1-processing-n:127.0.0.1:64804_solr) [n:127.0.0.1:64804_solr     ] o.a.s.c.SolrCore [collection49_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
  2> 268378 INFO  (Thread-351) [n:127.0.0.1:64804_solr     ] o.a.s.c.SolrCore [collection49_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@4b4008ba
  2> 268381 INFO  (Thread-349) [n:127.0.0.1:64800_solr     ] o.a.s.c.SolrCore [collection49_shard1_replica_n2]  CLOSING SolrCore org.apache.solr.core.SolrCore@32b5ff92
  2> 269179 INFO  (jetty-closer-1917-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
  2> 269179 INFO  (jetty-closer-1917-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5f32de4d: rootName = solr_64806, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@1f177735
  2> 269398 INFO  (jetty-closer-1917-thread-6) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
  2> 269398 INFO  (jetty-closer-1917-thread-6) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@32e8b56a: rootName = solr_64805, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@7ce7d
  2> 269457 INFO  (jetty-closer-1917-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
  2> 269457 INFO  (jetty-closer-1917-thread-3) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@d62275f: rootName = solr_64803, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@1297a519
  2> 269788 WARN  (qtp730558419-1114) [n:127.0.0.1:64802_solr     ] o.a.s.m.SolrMetricManager Could not obtain lock to modify reporters registry: solr.core.collection49.shard3.replica_n8
  2> 270218 INFO  (searcherExecutor-798-thread-1-processing-n:127.0.0.1:64804_solr) [n:127.0.0.1:64804_solr     ] o.a.s.c.SolrCore [collection49_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
  2> 270237 INFO  (searcherExecutor-801-thread-1-processing-n:127.0.0.1:64800_solr) [n:127.0.0.1:64800_solr     ] o.a.s.c.SolrCore [collection49_shard1_replica_n2]  Registered new searcher autowarm time: 0 ms
  2> 270370 INFO  (Thread-350) [n:127.0.0.1:64806_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
  2> 270370 INFO  (Thread-350) [n:127.0.0.1:64806_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
  2> 270545 INFO  (Thread-350) [n:127.0.0.1:64806_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
  2> 270548 INFO  (Thread-350) [n:127.0.0.1:64806_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
  2> 270548 INFO  (Thread-350) [n:127.0.0.1:64806_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
  2> 270549 INFO  (Thread-350) [n:127.0.0.1:64806_solr     ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1680150472913059840
  2> 270563 INFO  (Thread-350) [n:127.0.0.1:64806_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
  2> 270629 INFO  (Thread-352) [n:127.0.0.1:64805_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
  2> 270629 INFO  (Thread-352) [n:127.0.0.1:64805_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
  2> 270697 INFO  (Thread-348) [n:127.0.0.1:64803_solr     ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
  2> 270697 INFO  (Thread-348) [n:127.0.0.1:64803_solr     ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
  2> 270790 INFO  (searcherExecutor-808-thread-1-processing-n:127.0.0.1:64806_solr) [n:127.0.0.1:64806_solr     ] o.a.s.c.SolrCore [collection49_shard2_replica_n4]  Registered new searcher autowarm time: 0 ms
  2> 270793 INFO  (Thread-350) [n:127.0.0.1:64806_solr     ] o.a.s.c.SolrCore [collection49_shard2_replica_n4]  CLOSING SolrCore org.apache.solr.core.SolrCore@61d5c13e
  2> 270825 INFO  (Thread-352) [n:127.0.0.1:64805_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
  2> 270828 INFO  (Thread-352) [n:127.0.0.1:64805_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
  2> 270829 INFO  (Thread-352) [n:127.0.0.1:64805_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
  2> 270829 INFO  (Thread-352) [n:127.0.0.1:64805_solr     ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1680150473206661120
  2> 270833 INFO  (Thread-352) [n:127.0.0.1:64805_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
  2> 270862 INFO  (Thread-348) [n:127.0.0.1:64803_solr     ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
  2> 270867 INFO  (Thread-348) [n:127.0.0.1:64803_solr     ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
  2> 270868 INFO  (Thread-348) [n:127.0.0.1:64803_solr     ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
  2> 270868 INFO  (Thread-348) [n:127.0.0.1:64803_solr     ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1680150473247555584
  2> 270874 INFO  (Thread-348) [n:127.0.0.1:64803_solr     ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
  2> 270958 INFO  (searcherExecutor-806-thread-1-processing-n:127.0.0.1:64805_solr) [n:127.0.0.1:64805_solr     ] o.a.s.c.SolrCore [collection49_shard3_replica_n10]  Registered new searcher autowarm time: 0 ms
  2> 270960 INFO  (Thread-352) [n:127.0.0.1:64805_solr     ] o.a.s.c.SolrCore [collection49_shard3_replica_n10]  CLOSING SolrCore org.apache.solr.core.SolrCore@22739ead
  2> 271006 INFO  (searcherExecutor-810-thread-1-processing-n:127.0.0.1:64803_solr) [n:127.0.0.1:64803_solr     ] o.a.s.c.SolrCore [collection49_shard2_replica_n6]  Registered new searcher autowarm time: 0 ms
  2> 271017 INFO  (Thread-348) [n:127.0.0.1:64803_solr     ] o.a.s.c.SolrCore [collection49_shard2_replica_n6]  CLOSING SolrCore org.apache.solr.core.SolrCore@3ccd04f6
  2> 271134 INFO  (searcherExecutor-808-thread-1-processing-n:127.0.0.1:64806_solr) [n:127.0.0.1:64806_solr     ] o.a.s.c.SolrCore [collection49_shard2_replica_n4]  Registered new searcher autowarm time: 0 ms
  2> 271158 INFO  (jetty-closer-1917-thread-4) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
  2> 271158 INFO  (jetty-closer-1917-thread-4) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@79b76741: rootName = solr_64802, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@22c738c4
  2> 271171 INFO  (searcherExecutor-806-thread-1-processing-n:127.0.0.1:64805_solr) [n:127.0.0.1:64805_solr     ] o.a.s.c.SolrCore [collection49_shard3_replica_n10]  Registered new searcher autowarm time: 0 ms
  2> 271186 INFO  (searcherExecutor-810-thread-1-processing-n:127.0.0.1:64803_solr) [n:127.0.0.1:64803_solr     ] o.a.s.c.SolrCore [collection49_shard2_replica_n6]  Registered new searcher autowarm time: 0 ms
  2> 272357 INFO  (Thread-351) [n:127.0.0.1:64804_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection49.shard1.replica_n1 tag=SolrCore@4b4008ba
  2> 272358 INFO  (Thread-351) [n:127.0.0.1:64804_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7073c98a: rootName = solr_64804, domain = solr.core.collection49.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection49.shard1.replica_n1/com.codahale.metrics.MetricRegistry@776124a0
  2> 273135 INFO  (jetty-closer-1917-thread-5) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
  2> 273135 INFO  (jetty-closer-1917-thread-5) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4449fbf: rootName = solr_64800, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@22c738c4
  2> 273298 INFO  (qtp730558419-1114) [n:127.0.0.1:64802_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection49.shard3.leader tag=SolrCore@1a0025ad
  2> 273298 INFO  (jetty-closer-1917-thread-4) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
  2> 273299 INFO  (jetty-closer-1917-thread-4) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@65337635: rootName = solr_64802, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@1da86212
  2> 273299 INFO  (jetty-closer-1917-thread-4) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
  2> 273522 INFO  (jetty-closer-1917-thread-4) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@254c94d2{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
  2> 273522 INFO  (jetty-closer-1917-thread-4) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@160c6c{/solr,null,UNAVAILABLE}
  2> 273522 INFO  (jetty-closer-1917-thread-4) [     ] o.e.j.s.session node0 Stopped scavenging
  2> 273992 INFO  (jetty-closer-1917-thread-6) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
  2> 273993 INFO  (jetty-closer-1917-thread-6) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@184bd4de: rootName = solr_64805, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@22c738c4
  2> 274274 INFO  (jetty-closer-1917-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
  2> 274275 INFO  (jetty-closer-1917-thread-3) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4c608644: rootName = solr_64803, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@22c738c4
  2> 274339 INFO  (jetty-closer-1917-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
  2> 274339 INFO  (jetty-closer-1917-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1173081: rootName = solr_64806, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@22c738c4
  2> 277214 INFO  (jetty-closer-1917-thread-5) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
  2> 277214 INFO  (jetty-closer-1917-thread-5) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@41772b11: rootName = solr_64800, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@1da86212
  2> 277215 INFO  (Thread-349) [n:127.0.0.1:64800_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection49.shard1.replica_n2 tag=SolrCore@32b5ff92
  2> 277218 INFO  (Thread-349) [n:127.0.0.1:64800_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@460a507f: rootName = solr_64800, domain = solr.core.collection49.shard1.replica_n2, service url = null, agent id = null] for registry solr.core.collection49.shard1.replica_n2/com.codahale.metrics.MetricRegistry@18cb2c01
  2> 277256 INFO  (Thread-350) [n:127.0.0.1:64806_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection49.shard2.replica_n4 tag=SolrCore@61d5c13e
  2> 277256 INFO  (Thread-350) [n:127.0.0.1:64806_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5e918af6: rootName = solr_64806, domain = solr.core.collection49.shard2.replica_n4, service url = null, agent id = null] for registry solr.core.collection49.shard2.replica_n4/com.codahale.metrics.MetricRegistry@50f58c57
  2> 277606 INFO  (qtp730558419-1114) [n:127.0.0.1:64802_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
  2> 277857 INFO  (jetty-closer-1917-thread-6) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
  2> 277857 INFO  (jetty-closer-1917-thread-6) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@21fc4a49: rootName = solr_64805, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@1da86212
  2> 277857 INFO  (jetty-closer-1917-thread-6) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
  2> 277858 INFO  (Thread-352) [n:127.0.0.1:64805_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection49.shard3.replica_n10 tag=SolrCore@22739ead
  2> 277859 INFO  (Thread-352) [n:127.0.0.1:64805_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6c5f2b89: rootName = solr_64805, domain = solr.core.collection49.shard3.replica_n10, service url = null, agent id = null] for registry solr.core.collection49.shard3.replica_n10/com.codahale.metrics.MetricRegistry@b0afdd0
  2> 277894 INFO  (jetty-closer-1917-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
  2> 277895 INFO  (jetty-closer-1917-thread-3) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5395b738: rootName = solr_64803, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@1da86212
  2> 277895 INFO  (jetty-closer-1917-thread-3) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
  2> 277895 INFO  (Thread-348) [n:127.0.0.1

[...truncated too long message...]

details, please refer to the Sass blog:
  https://sass-lang.com/blog/posts/7828841

Successfully installed sass-3.7.4
Successfully installed public_suffix-4.0.6
Successfully installed temple-0.8.2
Successfully installed asciidoctor-2.0.10
Successfully installed safe_yaml-1.0.5
Successfully installed rouge-1.11.1
Successfully installed pathutil-0.16.2
Successfully installed mercenary-0.3.6
Successfully installed liquid-4.0.3
Successfully installed kramdown-1.17.0
Successfully installed jekyll-watch-1.5.1
Successfully installed jekyll-sass-converter-1.5.2
Successfully installed colorator-1.1.0
Successfully installed addressable-2.7.0
Successfully installed concurrent-ruby-1.0.5-java
Successfully installed tilt-2.0.10
Successfully installed slim-4.0.1
Successfully installed jekyll-3.5.2
Successfully installed jekyll-asciidoc-3.0.0
25 gems installed

> Task :solr:solr-ref-guide:buildSiteJekyll
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.headius.backport9.modules.Modules (file:/Users/jenkins/.gradle/caches/modules-2/files-2.1/org.jruby/jruby-complete/9.2.9.0/8e44ce7a1417966d89957bd766a148601e28828b/jruby-complete-9.2.9.0.jar) to method sun.nio.ch.NativeThread.signal(long)
WARNING: Please consider reporting this to the maintainers of com.headius.backport9.modules.Modules
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Configuration file: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/solr-ref-guide/build/content/_config.yml
            Source: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/solr-ref-guide/build/content
       Destination: ../html-site
 Incremental build: disabled. Enable with --incremental
      Generating... 
                    done in 21.662 seconds.
 Auto-regeneration: disabled. Use --watch to enable.

> Task :solr:solr-ref-guide:buildSite
Processed 2459 links (1773 relative) to 3538 anchors in 245 files

> Task :solr:solr-ref-guide:check
> Task :solr:solrj:check
> Task :solr:test-framework:check
> Task :solr:webapp:check
> Task :lucene:analysis:common:check
> Task :lucene:analysis:icu:check
> Task :lucene:analysis:kuromoji:check
> Task :lucene:analysis:morfologik:check
> Task :lucene:analysis:nori:check
> Task :lucene:analysis:opennlp:check
> Task :lucene:analysis:phonetic:check
> Task :lucene:analysis:smartcn:check
> Task :lucene:analysis:stempel:check
> Task :solr:contrib:analysis-extras:check
> Task :solr:contrib:analytics:check
> Task :solr:contrib:clustering:check
> Task :solr:contrib:extraction:check
> Task :solr:contrib:jaegertracer-configurator:check
> Task :solr:contrib:langid:check
> Task :solr:contrib:ltr:check
> Task :solr:contrib:prometheus-exporter:check
> Task :solr:docker:package:check
The slowest tests (exceeding 500 ms) during this run:
  243.59s TestSolrConfigHandlerCloud.test (:solr:core)
  227.82s HdfsDirectoryTest.testRandomAccessWrites (:solr:core)
  154.75s TestDistributedSearch.test (:solr:core)
  138.82s BasicDistributedZkTest.test (:solr:core)
  101.94s HttpPartitionTest.test (:solr:core)
  99.81s TestSolrCachePerf.testGetPutCompute (:solr:core)
  88.90s TestDistributedStatsComponentCardinality.test (:solr:core)
  81.23s DistribCursorPagingTest.test (:solr:core)
  80.89s PeerSyncReplicationTest.test (:solr:core)
  79.79s TestStressLiveNodes.testStress (:solr:core)

ERROR: The following test(s) have failed:
  - org.apache.solr.handler.component.CustomHighlightComponentTest.test (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.handler.component.CustomHighlightComponentTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.handler.component.CustomHighlightComponentTest" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UseG1GC" -Ptests.seed=417611789BCE826C -Ptests.badapples=false -Ptests.file.encoding=ISO-8859-1

  - org.apache.solr.cloud.HttpPartitionWithTlogReplicasTest.test (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.HttpPartitionWithTlogReplicasTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.HttpPartitionWithTlogReplicasTest" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UseG1GC" -Ptests.seed=417611789BCE826C -Ptests.badapples=false -Ptests.file.encoding=ISO-8859-1

  - org.apache.solr.cloud.HttpPartitionOnCommitTest.test (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.HttpPartitionOnCommitTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.HttpPartitionOnCommitTest" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UseG1GC" -Ptests.seed=417611789BCE826C -Ptests.badapples=false -Ptests.file.encoding=ISO-8859-1


Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 1h 25m 4s
849 actionable tasks: 849 executed
Build step 'Invoke Gradle script' changed build result to SUCCESS
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2875)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2754)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2735)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3122)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to MacOSX VBOX
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:1001)
		at hudson.FilePath.act(FilePath.java:1070)
		at hudson.FilePath.act(FilePath.java:1059)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2733)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:270)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:753)
		at hudson.model.Build$BuildExecution.post2(Build.java:177)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:698)
		at hudson.model.Run.execute(Run.java:1919)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3124)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:375)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1072)
	at hudson.FilePath.act(FilePath.java:1059)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2733)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:270)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:753)
	at hudson.model.Build$BuildExecution.post2(Build.java:177)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:698)
	at hudson.model.Run.execute(Run.java:1919)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/*_pid*.log". Configuration error?
[Java] Sleeping for 5 seconds due to JENKINS-32191...
[Java] Parsing console log (workspace: '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX')
[Java] Successfully parsed console log
[Java] -> found 0 issues (skipped 0 duplicates)
[Java] Skipping post processing
[Java] No filter has been set, publishing all 0 issues
[Java] Using reference build 'Lucene-Solr-master-MacOSX #6612' to compute new, fixed, and outstanding issues
[Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
[Java] No quality gates have been set - skipping
[Java] Health report is disabled - skipping
[Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)
[Java] Attaching ResultAction with ID 'java' to run 'Lucene-Solr-master-MacOSX #6613'.
[Checks API] No suitable checks publisher found.
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
[Checks API] No suitable checks publisher found.
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS] Lucene-Solr-master-MacOSX (64bit/jdk-14.0.1) - Build # 6612 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6612/
Java: 64bit/jdk-14.0.1 -XX:-UseCompressedOops -XX:+UseG1GC

2 tests failed.
FAILED:  org.apache.solr.cloud.HttpPartitionOnCommitTest.test

Error Message:
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:54795/bi_gn/bw: Underlying core creation failed while creating collection: c8n_2x2_commits

Stack Trace:
org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:54795/bi_gn/bw: Underlying core creation failed while creating collection: c8n_2x2_commits
	at __randomizedtesting.SeedInfo.seed([67B80D0875C39E46:EFEC32D2DB3FF3BE]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:698)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:269)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:251)
	at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1290)
	at org.apache.solr.cloud.BasicDistributedZkTest.createCollection(BasicDistributedZkTest.java:809)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1755)
	at org.apache.solr.cloud.HttpPartitionOnCommitTest.multiShardTest(HttpPartitionOnCommitTest.java:76)
	at org.apache.solr.cloud.HttpPartitionOnCommitTest.test(HttpPartitionOnCommitTest.java:67)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1090)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1061)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientRetryTest.classMethod

Error Message:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.impl.CloudSolrClientRetryTest: 
   1) Thread[id=439, name=SessionTracker, state=TIMED_WAITING, group=TGRP-CloudSolrClientRetryTest]
        at java.base@14.0.1/java.lang.Thread.sleep(Native Method)
        at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:157)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.impl.CloudSolrClientRetryTest: 
   1) Thread[id=439, name=SessionTracker, state=TIMED_WAITING, group=TGRP-CloudSolrClientRetryTest]
        at java.base@14.0.1/java.lang.Thread.sleep(Native Method)
        at app//org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:157)
	at __randomizedtesting.SeedInfo.seed([67B80D0875C39E46]:0)




Build Log:
[...truncated 3737 lines...]
ERROR: The following test(s) have failed:
  - org.apache.solr.client.solrj.impl.CloudSolrClientRetryTest.classMethod (:solr:solrj)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/solrj/build/test-results/test/outputs/OUTPUT-org.apache.solr.client.solrj.impl.CloudSolrClientRetryTest.txt
    Reproduce with: gradlew :solr:solrj:test --tests "org.apache.solr.client.solrj.impl.CloudSolrClientRetryTest" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UseG1GC" -Ptests.seed=67B80D0875C39E46 -Ptests.badapples=false -Ptests.file.encoding=US-ASCII

  - org.apache.solr.cloud.HttpPartitionOnCommitTest.test (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.HttpPartitionOnCommitTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.HttpPartitionOnCommitTest" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UseG1GC" -Ptests.seed=67B80D0875C39E46 -Ptests.badapples=false -Ptests.file.encoding=US-ASCII


Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 1h 19m 11s
849 actionable tasks: 849 executed
Build step 'Invoke Gradle script' changed build result to SUCCESS
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2875)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2754)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2735)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3122)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to MacOSX VBOX
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:1001)
		at hudson.FilePath.act(FilePath.java:1070)
		at hudson.FilePath.act(FilePath.java:1059)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2733)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:270)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:753)
		at hudson.model.Build$BuildExecution.post2(Build.java:177)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:698)
		at hudson.model.Run.execute(Run.java:1919)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3124)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:375)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1072)
	at hudson.FilePath.act(FilePath.java:1059)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2733)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:270)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:753)
	at hudson.model.Build$BuildExecution.post2(Build.java:177)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:698)
	at hudson.model.Run.execute(Run.java:1919)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/*_pid*.log". Configuration error?
[Java] Sleeping for 5 seconds due to JENKINS-32191...
[Java] Parsing console log (workspace: '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX')
[Java] Successfully parsed console log
[Java] -> found 0 issues (skipped 0 duplicates)
[Java] Skipping post processing
[Java] No filter has been set, publishing all 0 issues
[Java] Using reference build 'Lucene-Solr-master-MacOSX #6611' to compute new, fixed, and outstanding issues
[Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
[Java] No quality gates have been set - skipping
[Java] Health report is disabled - skipping
[Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)
[Java] Attaching ResultAction with ID 'java' to run 'Lucene-Solr-master-MacOSX #6612'.
[Checks API] No suitable checks publisher found.
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
[Checks API] No suitable checks publisher found.
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)

[JENKINS-Experimental-GC] Lucene-Solr-master-MacOSX (64bit/jdk-14.0.1) - Build # 6611 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-MacOSX/6611/
Java: 64bit/jdk-14.0.1 -XX:-UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC

5 tests failed.
FAILED:  org.apache.solr.cloud.DistributedQueueTest.testDistributedQueueBlocking

Error Message:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:49231/solr within 10000 ms

Stack Trace:
java.util.concurrent.TimeoutException: Could not connect to ZooKeeper 127.0.0.1:49231/solr within 10000 ms
	at __randomizedtesting.SeedInfo.seed([34795E62A3791560:71D32C1BE72BA914]:0)
	at org.apache.solr.common.cloud.ConnectionManager.waitForConnected(ConnectionManager.java:251)
	at org.apache.solr.cloud.DistributedQueueTest.forceSessionExpire(DistributedQueueTest.java:285)
	at org.apache.solr.cloud.DistributedQueueTest.testDistributedQueueBlocking(DistributedQueueTest.java:161)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.cloud.ReplaceNodeTest.test

Error Message:
java.lang.AssertionError

Stack Trace:
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([34795E62A3791560:BC2D61B80D857898]:0)
	at org.junit.Assert.fail(Assert.java:86)
	at org.junit.Assert.assertTrue(Assert.java:41)
	at org.junit.Assert.assertTrue(Assert.java:52)
	at org.apache.solr.cloud.ReplaceNodeTest.test(ReplaceNodeTest.java:108)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:942)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:978)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.cloud.TestSegmentSorting.testAtomicUpdateOfSegmentSortField

Error Message:
java.lang.AssertionError: expected:<COMPLETED> but was:<RUNNING>

Stack Trace:
java.lang.AssertionError: expected:<COMPLETED> but was:<RUNNING>
	at __randomizedtesting.SeedInfo.seed([34795E62A3791560:D56B7967753C8ED8]:0)
	at org.junit.Assert.fail(Assert.java:88)
	at org.junit.Assert.failNotEquals(Assert.java:834)
	at org.junit.Assert.assertEquals(Assert.java:118)
	at org.junit.Assert.assertEquals(Assert.java:144)
	at org.apache.solr.cloud.TestSegmentSorting.createCollection(TestSegmentSorting.java:85)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1754)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:976)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:819)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:470)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:951)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:887)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:898)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:370)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:826)
	at java.base/java.lang.Thread.run(Thread.java:832)


FAILED:  org.apache.solr.schema.TestCloudSchemaless.test

Error Message:
java.lang.Exception: Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([34795E62A3791560]:0)


FAILED:  org.apache.solr.schema.TestCloudSchemaless.classMethod

Error Message:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([34795E62A3791560]:0)




Build Log:
[...truncated 67085 lines...]
ERROR: The following test(s) have failed:
  - org.apache.solr.cloud.ReplaceNodeTest.test (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.ReplaceNodeTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.ReplaceNodeTest" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC" -Ptests.seed=34795E62A3791560 -Ptests.badapples=false -Ptests.file.encoding=US-ASCII

  - org.apache.solr.cloud.TestSegmentSorting.testAtomicUpdateOfSegmentSortField (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.TestSegmentSorting.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.TestSegmentSorting" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC" -Ptests.seed=34795E62A3791560 -Ptests.badapples=false -Ptests.file.encoding=US-ASCII

  - org.apache.solr.cloud.DistributedQueueTest.testDistributedQueueBlocking (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.DistributedQueueTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.DistributedQueueTest" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC" -Ptests.seed=34795E62A3791560 -Ptests.badapples=false -Ptests.file.encoding=US-ASCII

  - org.apache.solr.schema.TestCloudSchemaless.test (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.schema.TestCloudSchemaless.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.schema.TestCloudSchemaless" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC" -Ptests.seed=34795E62A3791560 -Ptests.badapples=false -Ptests.file.encoding=US-ASCII

  - org.apache.solr.schema.TestCloudSchemaless.classMethod (:solr:core)
    Test output: /Users/jenkins/workspace/Lucene-Solr-master-MacOSX/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.schema.TestCloudSchemaless.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.schema.TestCloudSchemaless" -Ptests.jvms=5 -Ptests.haltonfailure=false "-Ptests.jvmargs=-XX:-UseCompressedOops -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC" -Ptests.seed=34795E62A3791560 -Ptests.badapples=false -Ptests.file.encoding=US-ASCII


Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 3h 15m 39s
849 actionable tasks: 849 executed
Build step 'Invoke Gradle script' changed build result to SUCCESS
Archiving artifacts
java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath$ValidateAntFileMask.hasMatch(FilePath.java:2875)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2754)
	at hudson.FilePath$ValidateAntFileMask.invoke(FilePath.java:2735)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3122)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to MacOSX VBOX
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1800)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:1001)
		at hudson.FilePath.act(FilePath.java:1070)
		at hudson.FilePath.act(FilePath.java:1059)
		at hudson.FilePath.validateAntFileMask(FilePath.java:2733)
		at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:270)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:753)
		at hudson.model.Build$BuildExecution.post2(Build.java:177)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:698)
		at hudson.model.Run.execute(Run.java:1919)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:428)
Caused: hudson.FilePath$TunneledInterruptedException
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3124)
	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:375)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:73)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused: java.lang.InterruptedException: java.lang.InterruptedException: no matches found within 10000
	at hudson.FilePath.act(FilePath.java:1072)
	at hudson.FilePath.act(FilePath.java:1059)
	at hudson.FilePath.validateAntFileMask(FilePath.java:2733)
	at hudson.tasks.ArtifactArchiver.perform(ArtifactArchiver.java:270)
	at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
	at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:753)
	at hudson.model.Build$BuildExecution.post2(Build.java:177)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:698)
	at hudson.model.Run.execute(Run.java:1919)
	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:428)
No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/*_pid*.log". Configuration error?
[Java] Sleeping for 5 seconds due to JENKINS-32191...
[Java] Parsing console log (workspace: '/Users/jenkins/workspace/Lucene-Solr-master-MacOSX')
[Java] Successfully parsed console log
[Java] -> found 0 issues (skipped 0 duplicates)
[Java] Skipping post processing
[Java] No filter has been set, publishing all 0 issues
[Java] Using reference build 'Lucene-Solr-master-MacOSX #6610' to compute new, fixed, and outstanding issues
[Java] Issues delta (vs. reference build): outstanding: 0, new: 0, fixed: 0
[Java] No quality gates have been set - skipping
[Java] Health report is disabled - skipping
[Java] Created analysis result for 0 issues (found 0 new issues, fixed 0 issues)
[Java] Attaching ResultAction with ID 'java' to run 'Lucene-Solr-master-MacOSX #6611'.
[Checks API] No suitable checks publisher found.
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
[Checks API] No suitable checks publisher found.
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)