You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@solr.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2023/07/17 17:59:48 UTC

[JENKINS] Solr » Solr-Check-9.x - Build # 4901 - Still Failing!

Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4901/

2 tests failed.
FAILED:  org.apache.solr.ltr.TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder

Error Message:
java.lang.Exception: Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([D35C879180C1E1C8]:0)


FAILED:  org.apache.solr.ltr.TestParallelWeightCreation.classMethod

Error Message:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([D35C879180C1E1C8]:0)




Build Log:
[...truncated 1237 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22945ms
ok 43 post solr log into solr in 9208ms
ok 44 start solr with ssl in 19768ms
ok 45 start solr with ssl and auth in 22915ms
ok 46 SOLR-11740 check 'solr stop' connection in 28430ms
ok 47 status detects locally running solr in 14431ms
ok 48 status does not expose cli parameters to end user in 213ms
ok 49 listing out files in 2172ms
ok 50 copying files around in 6572ms
ok 51 upconfig in 2651ms

> Task :solr:packaging:integrationTests FAILED

> Task :solr:modules:ltr:test

org.apache.solr.ltr.TestParallelWeightCreation > testLTRScoringQueryParallelWeightCreationResultOrder FAILED
    java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([D35C879180C1E1C8]:0)

org.apache.solr.ltr.TestParallelWeightCreation > classMethod FAILED
    java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([D35C879180C1E1C8]:0)

org.apache.solr.ltr.TestParallelWeightCreation > test suite's output saved to /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.TestParallelWeightCreation.txt, copied below:
  1> 15:59:15.341 [TEST-TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder-seed#[D35C879180C1E1C8]] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
  2> suoi 18, 2023 3:59:13 IĐITBEAIVET com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
  2> WARNING: Suite execution timed out: org.apache.solr.ltr.TestParallelWeightCreation
  2> ==== jstack at approximately timeout time ====
  2> "Test worker" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$2@73833da0
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$2@73833da0
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1300)
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1375)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:647)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:504)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:108)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:57)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:39)
  2> 	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
  2> 	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:52)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
  2> 	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
  2> 	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
  2> 	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
  2> 	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
  2> 
  2> "Reference Handler" ID=2 RUNNABLE
  2> 	at java.base@11.0.16/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
  2> 	at java.base@11.0.16/java.lang.ref.Reference.processPendingReferences(Reference.java:241)
  2> 	at java.base@11.0.16/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)
  2> 
  2> "Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@177d48a
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.ref.ReferenceQueue$Lock@177d48a
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
  2> 	at java.base@11.0.16/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)
  2> 
  2> "Signal Dispatcher" ID=4 RUNNABLE
  2> 
  2> "Common-Cleaner" ID=9 TIMED_WAITING on java.lang.ref.ReferenceQueue$Lock@24034994
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- timed waiting on java.lang.ref.ReferenceQueue$Lock@24034994
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	at java.base@11.0.16/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:161)
  2> 
  2> "/127.0.0.1:48366 to /127.0.0.1:33171 workers" ID=11 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@15cb2559
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@15cb2559
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:403)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@42e99e4a
  2> 
  2> "/127.0.0.1:48366 to /127.0.0.1:33171 workers Thread 2" ID=12 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@14f642dc
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@14f642dc
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionDispatch.run(MessageHub.java:322)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@42a48628
  2> 
  2> "/127.0.0.1:48366 to /127.0.0.1:33171 workers Thread 3" ID=13 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@2b9a174c
  2> 	- locked sun.nio.ch.EPollSelectorImpl@15eba442
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at org.gradle.internal.remote.internal.inet.SocketConnection$SocketInputStream.read(SocketConnection.java:185)
  2> 	at com.esotericsoftware.kryo.io.Input.fill(Input.java:146)
  2> 	at com.esotericsoftware.kryo.io.Input.require(Input.java:178)
  2> 	at com.esotericsoftware.kryo.io.Input.readByte(Input.java:295)
  2> 	at org.gradle.internal.serialize.kryo.KryoBackedDecoder.readByte(KryoBackedDecoder.java:88)
  2> 	at org.gradle.internal.remote.internal.hub.InterHubMessageSerializer$MessageReader.read(InterHubMessageSerializer.java:64)
  2> 	at org.gradle.internal.remote.internal.hub.InterHubMessageSerializer$MessageReader.read(InterHubMessageSerializer.java:52)
  2> 	at org.gradle.internal.remote.internal.inet.SocketConnection.receive(SocketConnection.java:81)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionReceive.run(MessageHub.java:270)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@74f0ea28
  2> 
  2> "SUITE-TestParallelWeightCreation-seed#[D35C879180C1E1C8]" ID=14 RUNNABLE
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpThreads0(Native Method)
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:521)
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:509)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:708)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:59)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:437)
  2> 	- locked java.lang.Object@59c6ba4b
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:716)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:637)
  2> 
  2> "TEST-TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder-seed#[D35C879180C1E1C8]" ID=15 WAITING on java.util.concurrent.CountDownLatch$Sync@11e3b5e
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.CountDownLatch$Sync@11e3b5e
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
  2> 	at java.base@11.0.16/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
  2> 	at app//org.apache.solr.servlet.CoreContainerProvider$ContextInitializationKey.waitForReadyService(CoreContainerProvider.java:525)
  2> 	at app//org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:564)
  2> 	at app//org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
  2> 	at app//org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992)
  2> 	at app//org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427)
  2> 	- locked org.apache.solr.embedded.JettySolrRunner$1@7f80f30d
  2> 	at app//org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253)
  2> 	at app//org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554)
  2> 	- locked org.apache.solr.embedded.JettySolrRunner@4dbbbc6
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525)
  2> 	at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126)
  2> 	at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86)
  2> 	at app//org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57)
  2> 	at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181)
  2> 	at app//org.apache.solr.ltr.TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder(TestParallelWeightCreation.java:26)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2> 	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$$Lambda$177/0x00000008401edc40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@4cdad6ab
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@20a22999
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@5842ae1b
  2> 
  2> "qtp721800570-17" ID=17 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@1bebca58
  2> 	- locked sun.nio.ch.EPollSelectorImpl@2a34c65d
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034d040.run(Unknown Source)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp721800570-18-acceptor-0@126ee5c3-ServerConnector@54e13b4{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:43015}" ID=18 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
  2> 	at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
  2> 	at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@67cb8776
  2> 
  2> "qtp721800570-19" ID=19 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@177f941a
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@177f941a
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp721800570-20" ID=20 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@177f941a
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@177f941a
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp721800570-21" ID=21 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@177f941a
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@177f941a
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp721800570-22" ID=22 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@177f941a
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@177f941a
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp721800570-23" ID=23 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@177f941a
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@177f941a
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp721800570-24" ID=24 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@177f941a
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@177f941a
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "Session-HouseKeeper-62bb7916-1" ID=25 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@528335a7
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@528335a7
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
  2> 	at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "httpShardExecutor-6-thread-1" ID=26 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@5e090784
  2> 	- locked sun.nio.ch.EPollSelectorImpl@6542cb04
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034d040.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x0000000840441c40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@30e0d022
  2> 
  2> "httpShardExecutor-6-thread-2" ID=27 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@ef64cf2
  2> 	- locked sun.nio.ch.EPollSelectorImpl@45137e86
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034d040.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x0000000840441c40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@31c8c2ae
  2> 
  2> "Connection evictor" ID=28 TIMED_WAITING
  2> 	at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
  2> 	at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "Connection evictor" ID=29 TIMED_WAITING
  2> 	at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
  2> 	at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "h2sc-8-thread-1" ID=30 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@53306487
  2> 	- locked sun.nio.ch.EPollSelectorImpl@20d99f0e
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034d040.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x0000000840441c40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@4ce20f51
  2> 
  2> "h2sc-8-thread-2" ID=31 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@1c74c63b
  2> 	- locked sun.nio.ch.EPollSelectorImpl@424d7c7d
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034d040.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x0000000840441c40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@39ef3092
  2> 
  2> "CloserThread" ID=33 WAITING on java.lang.Object@12893cef
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.Object@12893cef
  2> 	at java.base@11.0.16/java.lang.Object.wait(Object.java:328)
  2> 	at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2564)
  2> 
  2> "ForkJoinPool.commonPool-worker-3" ID=36 WAITING on java.util.concurrent.ForkJoinPool@7ec4e05f
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.ForkJoinPool@7ec4e05f
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
  2> 	at java.base@11.0.16/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
  2> 
  2> "searcherExecutor-13-thread-1" ID=37 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@44bba4a2
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@44bba4a2
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> ^^==============================================
  2> 
   >     java.lang.Exception: Test abandoned because suite timeout was reached.
   >         at __randomizedtesting.SeedInfo.seed([D35C879180C1E1C8]:0)
  2> NOTE: reproduce with: gradlew test --tests TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder -Dtests.seed=D35C879180C1E1C8 -Dtests.locale=se -Dtests.timezone=Australia/Lindeman -Dtests.asserts=true -Dtests.file.encoding=UTF-8
  2> suoi 18, 2023 3:59:13 IĐITBEAIVET com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> WARNING: Will linger awaiting termination of 16 leaked thread(s).
  2> suoi 18, 2023 3:59:14 IĐITBEAIVET com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> SEVERE: 16 threads leaked from SUITE scope at org.apache.solr.ltr.TestParallelWeightCreation: 
  2>    1) Thread[id=23, name=qtp721800570-23, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    2) Thread[id=15, name=TEST-TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder-seed#[D35C879180C1E1C8], state=WAITING, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
  2>         at java.base@11.0.16/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
  2>         at app//org.apache.solr.servlet.CoreContainerProvider$ContextInitializationKey.waitForReadyService(CoreContainerProvider.java:525)
  2>         at app//org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:564)
  2>         at app//org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
  2>         at app//org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992)
  2>         at app//org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480)
  2>         at app//org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427)
  2>         at app//org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253)
  2>         at app//org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525)
  2>         at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126)
  2>         at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86)
  2>         at app//org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57)
  2>         at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181)
  2>         at app//org.apache.solr.ltr.TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder(TestParallelWeightCreation.java:26)
  2>         at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2>         at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2>         at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2>         at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  2>         at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2>         at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  2>         at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2>         at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$$Lambda$177/0x00000008401edc40.run(Unknown Source)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    3) Thread[id=19, name=qtp721800570-19, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    4) Thread[id=18, name=qtp721800570-18-acceptor-0@126ee5c3-ServerConnector@54e13b4{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:43015}, state=RUNNABLE, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
  2>         at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
  2>         at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    5) Thread[id=20, name=qtp721800570-20, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    6) Thread[id=26, name=httpShardExecutor-6-thread-1, state=RUNNABLE, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034d040.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x0000000840441c40.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    7) Thread[id=21, name=qtp721800570-21, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    8) Thread[id=24, name=qtp721800570-24, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    9) Thread[id=27, name=httpShardExecutor-6-thread-2, state=RUNNABLE, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034d040.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x0000000840441c40.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   10) Thread[id=33, name=CloserThread, state=WAITING, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2>         at java.base@11.0.16/java.lang.Object.wait(Object.java:328)
  2>         at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2564)
  2>   11) Thread[id=25, name=Session-HouseKeeper-62bb7916-1, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
  2>         at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   12) Thread[id=30, name=h2sc-8-thread-1, state=RUNNABLE, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034d040.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x0000000840441c40.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   13) Thread[id=22, name=qtp721800570-22, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   14) Thread[id=31, name=h2sc-8-thread-2, state=RUNNABLE, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034d040.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x0000000840441c40.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   15) Thread[id=17, name=qtp721800570-17, state=RUNNABLE, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034d040.run(Unknown Source)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   16) Thread[id=37, name=searcherExecutor-13-thread-1, state=WAITING, group=TGRP-TestParallelWeightCreation]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2>         at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> suoi 18, 2023 3:59:14 IĐITBEAIVET com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFO: Starting to interrupt leaked threads:
  2>    1) Thread[id=23, name=qtp721800570-23, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>    2) Thread[id=15, name=TEST-TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder-seed#[D35C879180C1E1C8], state=WAITING, group=TGRP-TestParallelWeightCreation]
  2>    3) Thread[id=19, name=qtp721800570-19, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>    4) Thread[id=18, name=qtp721800570-18-acceptor-0@126ee5c3-ServerConnector@54e13b4{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:43015}, state=RUNNABLE, group=TGRP-TestParallelWeightCreation]
  2>    5) Thread[id=20, name=qtp721800570-20, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>    6) Thread[id=26, name=httpShardExecutor-6-thread-1, state=RUNNABLE, group=TGRP-TestParallelWeightCreation]
  2>    7) Thread[id=21, name=qtp721800570-21, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>    8) Thread[id=24, name=qtp721800570-24, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>    9) Thread[id=27, name=httpShardExecutor-6-thread-2, state=RUNNABLE, group=TGRP-TestParallelWeightCreation]
  2>   10) Thread[id=33, name=CloserThread, state=WAITING, group=TGRP-TestParallelWeightCreation]
  2>   11) Thread[id=25, name=Session-HouseKeeper-62bb7916-1, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>   12) Thread[id=30, name=h2sc-8-thread-1, state=RUNNABLE, group=TGRP-TestParallelWeightCreation]
  2>   13) Thread[id=22, name=qtp721800570-22, state=TIMED_WAITING, group=TGRP-TestParallelWeightCreation]
  2>   14) Thread[id=31, name=h2sc-8-thread-2, state=RUNNABLE, group=TGRP-TestParallelWeightCreation]
  2>   15) Thread[id=17, name=qtp721800570-17, state=RUNNABLE, group=TGRP-TestParallelWeightCreation]
  2>   16) Thread[id=37, name=searcherExecutor-13-thread-1, state=WAITING, group=TGRP-TestParallelWeightCreation]
  1> 17:59:14.855 [TEST-TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder-seed#[D35C879180C1E1C8]] ERROR org.apache.solr.servlet.SolrDispatchFilter - Could not start Dispatch Filter.
  1> org.apache.solr.common.SolrException: Interrupted while obtaining reference to CoreService
  1> 	at org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:571) ~[solr-core-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 110d45b7c93bbeac5faf87a9d0f40a4c8462d848 [snapshot build, details omitted]]
  1> 	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147) [solr-core-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 110d45b7c93bbeac5faf87a9d0f40a4c8462d848 [snapshot build, details omitted]]
  1> 	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 110d45b7c93bbeac5faf87a9d0f40a4c8462d848 [snapshot build, details omitted]]
  1> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253) [jetty-util-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94) [jetty-util-10.0.15.jar:10.0.15]
  1> 	at org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 110d45b7c93bbeac5faf87a9d0f40a4c8462d848 [snapshot build, details omitted]]
  1> 	at org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 110d45b7c93bbeac5faf87a9d0f40a4c8462d848 [snapshot build, details omitted]]
  1> 	at org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 110d45b7c93bbeac5faf87a9d0f40a4c8462d848 [snapshot build, details omitted]]
  1> 	at org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 110d45b7c93bbeac5faf87a9d0f40a4c8462d848 [snapshot build, details omitted]]
  1> 	at org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 110d45b7c93bbeac5faf87a9d0f40a4c8462d848 [snapshot build, details omitted]]
  1> 	at org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 110d45b7c93bbeac5faf87a9d0f40a4c8462d848 [snapshot build, details omitted]]
  1> 	at org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181) [test/:?]
  1> 	at org.apache.solr.ltr.TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder(TestParallelWeightCreation.java:26) [test/:?]
  1> 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
  1> 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
  1> 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
  1> 	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at java.lang.Thread.run(Thread.java:829) [?:?]
  2> NOTE: leaving temporary files on disk at: /tmp/src/solr/solr/modules/ltr/build/tmp/tests-tmp/solr.ltr.TestParallelWeightCreation_D35C879180C1E1C8-001
  2> Jul 17, 2023 5:59:15 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFO: All leaked threads terminated.
   >     java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   >         at __randomizedtesting.SeedInfo.seed([D35C879180C1E1C8]:0)
  2> NOTE: test params are: codec=HighCompressionCompressingStoredFieldsData(storedFieldsFormat=Lucene90CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=8025, maxDocsPerChunk=134, blockShift=10), termVectorsFormat=Lucene90CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=8025, maxDocsPerChunk=134, blockSize=10)), sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=se, timezone=Australia/Lindeman
  2> NOTE: Linux 5.4.0-100-generic amd64/Oracle Corporation 11.0.16 (64-bit)/cpus=1,threads=1,free=252538240,total=324534272
  2> NOTE: All tests run in this JVM: [TestParallelWeightCreation]
  2> NOTE: reproduce with: gradlew test --tests TestParallelWeightCreation -Dtests.seed=D35C879180C1E1C8 -Dtests.locale=se -Dtests.timezone=Australia/Lindeman -Dtests.asserts=true -Dtests.file.encoding=UTF-8

:solr:modules:ltr:test (FAILURE): 186 test(s), 2 failure(s)

186 tests completed, 2 failed

> Task :solr:modules:ltr:test FAILED
> Task :solr:modules:ltr:wipeTaskTemp

ERROR: The following test(s) have failed:
  - org.apache.solr.ltr.TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.TestParallelWeightCreation.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.TestParallelWeightCreation.testLTRScoringQueryParallelWeightCreationResultOrder" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=D35C879180C1E1C8 -Ptests.file.encoding=UTF-8

  - org.apache.solr.ltr.TestParallelWeightCreation.classMethod (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.TestParallelWeightCreation.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.TestParallelWeightCreation.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=D35C879180C1E1C8 -Ptests.file.encoding=UTF-8


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:modules:ltr:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/modules/ltr/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 2h 2m 56s
642 actionable tasks: 626 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 2h3m19.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4965 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4965/

5 tests failed.
FAILED:  org.apache.solr.search.function.TestDenseVectorFunctionQuery.byteFieldVectors_shouldReturnFloatSimilarity

Error Message:
java.lang.RuntimeException: Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([C4AC7AAB095B5664:3487AA7667081A6D]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:1024)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:982)
	at org.apache.solr.search.function.TestDenseVectorFunctionQuery.byteFieldVectors_shouldReturnFloatSimilarity(TestDenseVectorFunctionQuery.java:126)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: no byte vector value is indexed for field 'vector_byte_encoding'
	at org.apache.lucene.queries.function.valuesource.ByteKnnVectorFieldSource.getValues(ByteKnnVectorFieldSource.java:45)
	at org.apache.lucene.queries.function.valuesource.VectorSimilarityFunction.getValues(VectorSimilarityFunction.java:47)
	at org.apache.lucene.queries.function.FunctionQuery$AllScorer.<init>(FunctionQuery.java:115)
	at org.apache.lucene.queries.function.FunctionQuery$FunctionWeight.scorer(FunctionQuery.java:76)
	at org.apache.lucene.search.Weight.scorerSupplier(Weight.java:136)
	at org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:542)
	at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:507)
	at org.apache.lucene.search.Weight.bulkScorer(Weight.java:166)
	at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:395)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:770)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:720)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:551)
	at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:275)
	at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1890)
	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1707)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:710)
	at org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1660)
	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:421)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:425)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2893)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:348)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:328)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:996)
	... 49 more


FAILED:  org.apache.solr.search.function.TestDenseVectorFunctionQuery.resultOfVectorFunction_canBeUsedAsFloatFunctionInput

Error Message:
java.lang.RuntimeException: Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([C4AC7AAB095B5664:1D2FF383EF6B0796]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:1024)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:982)
	at org.apache.solr.search.function.TestDenseVectorFunctionQuery.resultOfVectorFunction_canBeUsedAsFloatFunctionInput(TestDenseVectorFunctionQuery.java:142)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: no float vector value is indexed for field 'vector'
	at org.apache.lucene.queries.function.valuesource.FloatKnnVectorFieldSource.getValues(FloatKnnVectorFieldSource.java:45)
	at org.apache.lucene.queries.function.valuesource.VectorSimilarityFunction.getValues(VectorSimilarityFunction.java:48)
	at org.apache.lucene.queries.function.valuesource.DualFloatFunction.getValues(DualFloatFunction.java:58)
	at org.apache.lucene.queries.function.FunctionQuery$AllScorer.<init>(FunctionQuery.java:115)
	at org.apache.lucene.queries.function.FunctionQuery$FunctionWeight.scorer(FunctionQuery.java:76)
	at org.apache.lucene.search.Weight.scorerSupplier(Weight.java:136)
	at org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:542)
	at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:507)
	at org.apache.lucene.search.Weight.bulkScorer(Weight.java:166)
	at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:395)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:770)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:720)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:551)
	at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:275)
	at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1890)
	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1707)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:710)
	at org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1660)
	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:421)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:425)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2893)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:348)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:328)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:996)
	... 49 more


FAILED:  org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_missingFieldValue_shouldReturnSimilarityZero

Error Message:
java.lang.RuntimeException: Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([C4AC7AAB095B5664:5084310B53A6AF72]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:1024)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:982)
	at org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_missingFieldValue_shouldReturnSimilarityZero(TestDenseVectorFunctionQuery.java:173)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: no float vector value is indexed for field 'vector2'
	at org.apache.lucene.queries.function.valuesource.FloatKnnVectorFieldSource.getValues(FloatKnnVectorFieldSource.java:45)
	at org.apache.lucene.queries.function.valuesource.VectorSimilarityFunction.getValues(VectorSimilarityFunction.java:48)
	at org.apache.lucene.queries.function.FunctionQuery$AllScorer.<init>(FunctionQuery.java:115)
	at org.apache.lucene.queries.function.FunctionQuery$FunctionWeight.scorer(FunctionQuery.java:76)
	at org.apache.lucene.search.Weight.scorerSupplier(Weight.java:136)
	at org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:542)
	at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:507)
	at org.apache.lucene.search.Weight.bulkScorer(Weight.java:166)
	at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:395)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:770)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:720)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:551)
	at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:275)
	at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1890)
	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1707)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:710)
	at org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1660)
	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:421)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:425)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2893)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:348)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:328)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:996)
	... 49 more


FAILED:  org.apache.solr.search.function.TestDenseVectorFunctionQuery.byteFieldVectors_missingFieldValue_shouldReturnSimilarityZero

Error Message:
java.lang.RuntimeException: Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([C4AC7AAB095B5664:1ACC3888998BB898]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:1024)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:982)
	at org.apache.solr.search.function.TestDenseVectorFunctionQuery.byteFieldVectors_missingFieldValue_shouldReturnSimilarityZero(TestDenseVectorFunctionQuery.java:157)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: no byte vector value is indexed for field 'vector_byte_encoding'
	at org.apache.lucene.queries.function.valuesource.ByteKnnVectorFieldSource.getValues(ByteKnnVectorFieldSource.java:45)
	at org.apache.lucene.queries.function.valuesource.VectorSimilarityFunction.getValues(VectorSimilarityFunction.java:48)
	at org.apache.lucene.queries.function.FunctionQuery$AllScorer.<init>(FunctionQuery.java:115)
	at org.apache.lucene.queries.function.FunctionQuery$FunctionWeight.scorer(FunctionQuery.java:76)
	at org.apache.lucene.search.Weight.scorerSupplier(Weight.java:136)
	at org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:542)
	at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:507)
	at org.apache.lucene.search.Weight.bulkScorer(Weight.java:166)
	at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:395)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:770)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:720)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:551)
	at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:275)
	at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1890)
	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1707)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:710)
	at org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1660)
	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:421)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:425)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2893)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:348)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:328)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:996)
	... 49 more


FAILED:  org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_shouldReturnFloatSimilarity

Error Message:
java.lang.RuntimeException: Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([C4AC7AAB095B5664:8A1EE5C66B20F640]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:1024)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:982)
	at org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_shouldReturnFloatSimilarity(TestDenseVectorFunctionQuery.java:111)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: no float vector value is indexed for field 'vector'
	at org.apache.lucene.queries.function.valuesource.FloatKnnVectorFieldSource.getValues(FloatKnnVectorFieldSource.java:45)
	at org.apache.lucene.queries.function.valuesource.VectorSimilarityFunction.getValues(VectorSimilarityFunction.java:47)
	at org.apache.lucene.queries.function.FunctionQuery$AllScorer.<init>(FunctionQuery.java:115)
	at org.apache.lucene.queries.function.FunctionQuery$FunctionWeight.scorer(FunctionQuery.java:76)
	at org.apache.lucene.search.Weight.scorerSupplier(Weight.java:136)
	at org.apache.lucene.search.BooleanWeight.scorerSupplier(BooleanWeight.java:542)
	at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:507)
	at org.apache.lucene.search.Weight.bulkScorer(Weight.java:166)
	at org.apache.lucene.search.BooleanWeight.bulkScorer(BooleanWeight.java:395)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:770)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:720)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:551)
	at org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:275)
	at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1890)
	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1707)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:710)
	at org.apache.solr.handler.component.QueryComponent.doProcessUngroupedSearch(QueryComponent.java:1660)
	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:421)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:425)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2893)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:348)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:328)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:996)
	... 49 more




Build Log:
[...truncated 3212 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22934ms
ok 43 post solr log into solr in 9211ms
ok 44 start solr with ssl in 20104ms
ok 45 start solr with ssl and auth in 23295ms
ok 46 SOLR-11740 check 'solr stop' connection in 28180ms
ok 47 status detects locally running solr in 14454ms
ok 48 status does not expose cli parameters to end user in 214ms
ok 49 listing out files in 2194ms
ok 50 copying files around in 6640ms
ok 51 upconfig in 2691ms

> Task :solr:packaging:integrationTests FAILED

ERROR: The following test(s) have failed:
  - org.apache.solr.search.function.TestDenseVectorFunctionQuery.byteFieldVectors_shouldReturnFloatSimilarity (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.function.TestDenseVectorFunctionQuery.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.function.TestDenseVectorFunctionQuery.byteFieldVectors_shouldReturnFloatSimilarity" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=C4AC7AAB095B5664 -Ptests.file.encoding=ISO-8859-1

  - org.apache.solr.search.function.TestDenseVectorFunctionQuery.resultOfVectorFunction_canBeUsedAsFloatFunctionInput (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.function.TestDenseVectorFunctionQuery.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.function.TestDenseVectorFunctionQuery.resultOfVectorFunction_canBeUsedAsFloatFunctionInput" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=C4AC7AAB095B5664 -Ptests.file.encoding=ISO-8859-1

  - org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_missingFieldValue_shouldReturnSimilarityZero (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.function.TestDenseVectorFunctionQuery.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_missingFieldValue_shouldReturnSimilarityZero" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=C4AC7AAB095B5664 -Ptests.file.encoding=ISO-8859-1

  - org.apache.solr.search.function.TestDenseVectorFunctionQuery.byteFieldVectors_missingFieldValue_shouldReturnSimilarityZero (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.function.TestDenseVectorFunctionQuery.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.function.TestDenseVectorFunctionQuery.byteFieldVectors_missingFieldValue_shouldReturnSimilarityZero" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=C4AC7AAB095B5664 -Ptests.file.encoding=ISO-8859-1

  - org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_shouldReturnFloatSimilarity (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.function.TestDenseVectorFunctionQuery.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.function.TestDenseVectorFunctionQuery.floatFieldVectors_shouldReturnFloatSimilarity" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=C4AC7AAB095B5664 -Ptests.file.encoding=ISO-8859-1


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:core:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/core/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 13m 19s
625 actionable tasks: 609 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 13m43.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4964 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4964/

All tests passed

Build Log:
[...truncated 1252 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22815ms
ok 43 post solr log into solr in 9493ms
ok 44 start solr with ssl in 20344ms
ok 45 start solr with ssl and auth in 23232ms
ok 46 SOLR-11740 check 'solr stop' connection in 28451ms
ok 47 status detects locally running solr in 14434ms
ok 48 status does not expose cli parameters to end user in 220ms
ok 49 listing out files in 2199ms
ok 50 copying files around in 6619ms
ok 51 upconfig in 2633ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 15m 58s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m23.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4963 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4963/

All tests passed

Build Log:
[...truncated 1258 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22606ms
ok 43 post solr log into solr in 9045ms
ok 44 start solr with ssl in 19762ms
ok 45 start solr with ssl and auth in 22215ms
ok 46 SOLR-11740 check 'solr stop' connection in 28467ms
ok 47 status detects locally running solr in 14379ms
ok 48 status does not expose cli parameters to end user in 217ms
ok 49 listing out files in 2175ms
ok 50 copying files around in 6615ms
ok 51 upconfig in 2647ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 15m 58s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m24.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4962 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4962/

All tests passed

Build Log:
[...truncated 1251 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22160ms
ok 43 post solr log into solr in 9344ms
ok 44 start solr with ssl in 20097ms
ok 45 start solr with ssl and auth in 22378ms
ok 46 SOLR-11740 check 'solr stop' connection in 28093ms
ok 47 status detects locally running solr in 14645ms
ok 48 status does not expose cli parameters to end user in 214ms
ok 49 listing out files in 2199ms
ok 50 copying files around in 6625ms
ok 51 upconfig in 2635ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 15m 54s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m21.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4961 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4961/

1 tests failed.
FAILED:  org.apache.solr.cloud.ReindexCollectionTest.testAbort

Error Message:
java.lang.AssertionError: {responseHeader={status=0, QTime=6}, status={state=running, msg=found [e2d24d76-a970-4ef1-bc01-b0cfd60b4760] in running tasks}}

Stack Trace:
java.lang.AssertionError: {responseHeader={status=0, QTime=6}, status={state=running, msg=found [e2d24d76-a970-4ef1-bc01-b0cfd60b4760] in running tasks}}
	at __randomizedtesting.SeedInfo.seed([49407C1A44F3BDB2:2F23CB8D30292FCB]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.assertTrue(Assert.java:42)
	at org.junit.Assert.assertNotNull(Assert.java:713)
	at org.apache.solr.cloud.ReindexCollectionTest.testAbort(ReindexCollectionTest.java:425)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)




Build Log:
[...truncated 6575 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22683ms
ok 43 post solr log into solr in 9697ms
ok 44 start solr with ssl in 20156ms
ok 45 start solr with ssl and auth in 22392ms
ok 46 SOLR-11740 check 'solr stop' connection in 28436ms
ok 47 status detects locally running solr in 14685ms
ok 48 status does not expose cli parameters to end user in 213ms
ok 49 listing out files in 2184ms
ok 50 copying files around in 6554ms
ok 51 upconfig in 2650ms

> Task :solr:packaging:integrationTests FAILED

ERROR: The following test(s) have failed:
  - org.apache.solr.cloud.ReindexCollectionTest.testAbort (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.ReindexCollectionTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.ReindexCollectionTest.testAbort" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=49407C1A44F3BDB2 -Ptests.file.encoding=US-ASCII


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:core:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/core/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 4s
625 actionable tasks: 609 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m30.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4960 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4960/

All tests passed

Build Log:
[...truncated 1255 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22749ms
ok 43 post solr log into solr in 9077ms
ok 44 start solr with ssl in 19903ms
ok 45 start solr with ssl and auth in 22821ms
ok 46 SOLR-11740 check 'solr stop' connection in 28124ms
ok 47 status detects locally running solr in 14356ms
ok 48 status does not expose cli parameters to end user in 214ms
ok 49 listing out files in 2176ms
ok 50 copying files around in 6553ms
ok 51 upconfig in 2653ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 13m 6s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 13m30s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4959 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4959/

All tests passed

Build Log:
[...truncated 1264 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22852ms
ok 43 post solr log into solr in 9220ms
ok 44 start solr with ssl in 20177ms
ok 45 start solr with ssl and auth in 23189ms
ok 46 SOLR-11740 check 'solr stop' connection in 28198ms
ok 47 status detects locally running solr in 14450ms
ok 48 status does not expose cli parameters to end user in 215ms
ok 49 listing out files in 2181ms
ok 50 copying files around in 6638ms
ok 51 upconfig in 2662ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 13m 9s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 13m34.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4958 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4958/

All tests passed

Build Log:
[...truncated 1262 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22465ms
ok 43 post solr log into solr in 9076ms
ok 44 start solr with ssl in 20082ms
ok 45 start solr with ssl and auth in 22675ms
ok 46 SOLR-11740 check 'solr stop' connection in 28747ms
ok 47 status detects locally running solr in 14451ms
ok 48 status does not expose cli parameters to end user in 216ms
ok 49 listing out files in 2204ms
ok 50 copying files around in 6582ms
ok 51 upconfig in 2634ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 13m 7s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 13m33.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4957 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4957/

All tests passed

Build Log:
[...truncated 1251 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22311ms
ok 43 post solr log into solr in 8823ms
ok 44 start solr with ssl in 20175ms
ok 45 start solr with ssl and auth in 23118ms
ok 46 SOLR-11740 check 'solr stop' connection in 28125ms
ok 47 status detects locally running solr in 14374ms
ok 48 status does not expose cli parameters to end user in 212ms
ok 49 listing out files in 2176ms
ok 50 copying files around in 6561ms
ok 51 upconfig in 2648ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 15m 52s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m16.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4956 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4956/

All tests passed

Build Log:
[...truncated 1246 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 23946ms
ok 43 post solr log into solr in 10287ms
ok 44 start solr with ssl in 21070ms
ok 45 start solr with ssl and auth in 22941ms
ok 46 SOLR-11740 check 'solr stop' connection in 28388ms
ok 47 status detects locally running solr in 14606ms
ok 48 status does not expose cli parameters to end user in 236ms
ok 49 listing out files in 2260ms
ok 50 copying files around in 7024ms
ok 51 upconfig in 2912ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 41s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 17m12s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4955 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4955/

1 tests failed.
FAILED:  org.apache.solr.metrics.DelegateRegistryTimerTest.testUpdate

Error Message:
java.lang.AssertionError: expected:<200.0> but was:<200.49874378911622>

Stack Trace:
java.lang.AssertionError: expected:<200.0> but was:<200.49874378911622>
	at __randomizedtesting.SeedInfo.seed([DF4E04C68C3DAA88:695841CCEAF9425E]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:555)
	at org.junit.Assert.assertEquals(Assert.java:685)
	at org.apache.solr.metrics.DelegateRegistryTimerTest.testUpdate(DelegateRegistryTimerTest.java:88)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)




Build Log:
[...truncated 1344 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22851ms
ok 43 post solr log into solr in 9464ms
ok 44 start solr with ssl in 20352ms
ok 45 start solr with ssl and auth in 22964ms
ok 46 SOLR-11740 check 'solr stop' connection in 28149ms
ok 47 status detects locally running solr in 14399ms
ok 48 status does not expose cli parameters to end user in 211ms
ok 49 listing out files in 2189ms
ok 50 copying files around in 6627ms
ok 51 upconfig in 2654ms

> Task :solr:packaging:integrationTests FAILED

ERROR: The following test(s) have failed:
  - org.apache.solr.metrics.DelegateRegistryTimerTest.testUpdate (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.metrics.DelegateRegistryTimerTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.metrics.DelegateRegistryTimerTest.testUpdate" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=DF4E04C68C3DAA88 -Ptests.file.encoding=US-ASCII


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:core:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/core/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 4s
625 actionable tasks: 609 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m28.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4954 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4954/

2 tests failed.
FAILED:  org.apache.solr.ltr.feature.TestFieldLengthFeature.testRankingDynamicField

Error Message:
java.lang.Exception: Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([11BD7105C2DBC451]:0)


FAILED:  org.apache.solr.ltr.feature.TestFieldLengthFeature.classMethod

Error Message:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([11BD7105C2DBC451]:0)




Build Log:
[...truncated 1245 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22780ms
ok 43 post solr log into solr in 9279ms
ok 44 start solr with ssl in 19786ms
ok 45 start solr with ssl and auth in 23298ms
ok 46 SOLR-11740 check 'solr stop' connection in 28060ms
ok 47 status detects locally running solr in 14405ms
ok 48 status does not expose cli parameters to end user in 210ms
ok 49 listing out files in 2171ms
ok 50 copying files around in 6609ms
ok 51 upconfig in 2687ms

> Task :solr:packaging:integrationTests FAILED

> Task :solr:modules:ltr:test

org.apache.solr.ltr.feature.TestFieldLengthFeature > testRankingDynamicField FAILED
    java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([11BD7105C2DBC451]:0)

org.apache.solr.ltr.feature.TestFieldLengthFeature > classMethod FAILED
    java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([11BD7105C2DBC451]:0)

org.apache.solr.ltr.feature.TestFieldLengthFeature > test suite's output saved to /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.feature.TestFieldLengthFeature.txt, copied below:
  1> 09:16:56.022 [TEST-TestFieldLengthFeature.testRankingDynamicField-seed#[11BD7105C2DBC451]] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
  2> ᎫᏰ 23, 2023 1:16:54 ᏒᎯᏱᎢᏗᏢ com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
  2> WARNING: Suite execution timed out: org.apache.solr.ltr.feature.TestFieldLengthFeature
  2> ==== jstack at approximately timeout time ====
  2> "Test worker" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$2@6e7e48fe
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$2@6e7e48fe
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1300)
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1375)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:647)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:504)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:108)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:57)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:39)
  2> 	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
  2> 	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:52)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
  2> 	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
  2> 	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
  2> 	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
  2> 	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
  2> 
  2> "Reference Handler" ID=2 RUNNABLE
  2> 	at java.base@11.0.16/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
  2> 	at java.base@11.0.16/java.lang.ref.Reference.processPendingReferences(Reference.java:241)
  2> 	at java.base@11.0.16/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)
  2> 
  2> "Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@799101d7
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.ref.ReferenceQueue$Lock@799101d7
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
  2> 	at java.base@11.0.16/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)
  2> 
  2> "Signal Dispatcher" ID=4 RUNNABLE
  2> 
  2> "Common-Cleaner" ID=9 TIMED_WAITING on java.lang.ref.ReferenceQueue$Lock@7d4ab11b
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- timed waiting on java.lang.ref.ReferenceQueue$Lock@7d4ab11b
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	at java.base@11.0.16/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:161)
  2> 
  2> "/127.0.0.1:49940 to /127.0.0.1:44657 workers" ID=11 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@271095e7
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@271095e7
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:403)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@42e99e4a
  2> 
  2> "/127.0.0.1:49940 to /127.0.0.1:44657 workers Thread 2" ID=12 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2de5feba
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2de5feba
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionDispatch.run(MessageHub.java:322)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@42a48628
  2> 
  2> "/127.0.0.1:49940 to /127.0.0.1:44657 workers Thread 3" ID=13 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@5c0bcd68
  2> 	- locked sun.nio.ch.EPollSelectorImpl@74e17d9b
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at org.gradle.internal.remote.internal.inet.SocketConnection$SocketInputStream.read(SocketConnection.java:185)
  2> 	at com.esotericsoftware.kryo.io.Input.fill(Input.java:146)
  2> 	at com.esotericsoftware.kryo.io.Input.require(Input.java:178)
  2> 	at com.esotericsoftware.kryo.io.Input.readByte(Input.java:295)
  2> 	at org.gradle.internal.serialize.kryo.KryoBackedDecoder.readByte(KryoBackedDecoder.java:88)
  2> 	at org.gradle.internal.remote.internal.hub.InterHubMessageSerializer$MessageReader.read(InterHubMessageSerializer.java:64)
  2> 	at org.gradle.internal.remote.internal.hub.InterHubMessageSerializer$MessageReader.read(InterHubMessageSerializer.java:52)
  2> 	at org.gradle.internal.remote.internal.inet.SocketConnection.receive(SocketConnection.java:81)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionReceive.run(MessageHub.java:270)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@74f0ea28
  2> 
  2> "SUITE-TestFieldLengthFeature-seed#[11BD7105C2DBC451]" ID=14 RUNNABLE
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpThreads0(Native Method)
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:521)
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:509)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:708)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:59)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:437)
  2> 	- locked java.lang.Object@577d714d
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:716)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:637)
  2> 
  2> "TEST-TestFieldLengthFeature.testRankingDynamicField-seed#[11BD7105C2DBC451]" ID=15 WAITING on java.util.concurrent.CountDownLatch$Sync@1791168
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.CountDownLatch$Sync@1791168
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
  2> 	at java.base@11.0.16/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
  2> 	at app//org.apache.solr.servlet.CoreContainerProvider$ContextInitializationKey.waitForReadyService(CoreContainerProvider.java:525)
  2> 	at app//org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:564)
  2> 	at app//org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
  2> 	at app//org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992)
  2> 	at app//org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427)
  2> 	- locked org.apache.solr.embedded.JettySolrRunner$1@55ef5536
  2> 	at app//org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253)
  2> 	at app//org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554)
  2> 	- locked org.apache.solr.embedded.JettySolrRunner@570db26e
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525)
  2> 	at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126)
  2> 	at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86)
  2> 	at app//org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57)
  2> 	at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181)
  2> 	at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:113)
  2> 	at app//org.apache.solr.ltr.feature.TestFieldLengthFeature.before(TestFieldLengthFeature.java:31)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:980)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2> 	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$$Lambda$177/0x00000008401edc40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@28ff00b1
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@1434ebfa
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@3b5d4aaf
  2> 
  2> "qtp1784879780-17" ID=17 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@5153737e
  2> 	- locked sun.nio.ch.EPollSelectorImpl@628b2995
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000840305c40.run(Unknown Source)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1784879780-18-acceptor-0@4bc46881-ServerConnector@47e867d{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:34395}" ID=18 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
  2> 	at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
  2> 	at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@33520f59
  2> 
  2> "qtp1784879780-19" ID=19 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ae9aa0a
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ae9aa0a
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1784879780-20" ID=20 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ae9aa0a
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ae9aa0a
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1784879780-21" ID=21 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ae9aa0a
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ae9aa0a
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1784879780-22" ID=22 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ae9aa0a
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ae9aa0a
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1784879780-23" ID=23 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ae9aa0a
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ae9aa0a
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1784879780-24" ID=24 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ae9aa0a
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7ae9aa0a
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "Session-HouseKeeper-41ed08f0-1" ID=25 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@30ebeeaa
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@30ebeeaa
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
  2> 	at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "httpShardExecutor-6-thread-1" ID=26 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@b97bea3
  2> 	- locked sun.nio.ch.EPollSelectorImpl@281c504f
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000840305c40.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@25d6c468
  2> 
  2> "httpShardExecutor-6-thread-2" ID=27 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@1f9206bc
  2> 	- locked sun.nio.ch.EPollSelectorImpl@70e3355a
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000840305c40.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@70ebeb3c
  2> 
  2> "Connection evictor" ID=28 TIMED_WAITING
  2> 	at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
  2> 	at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "Connection evictor" ID=29 TIMED_WAITING
  2> 	at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
  2> 	at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "h2sc-8-thread-1" ID=30 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@4637ff43
  2> 	- locked sun.nio.ch.EPollSelectorImpl@ce351e2
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000840305c40.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@3df56d7d
  2> 
  2> "h2sc-8-thread-2" ID=31 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@34b0124
  2> 	- locked sun.nio.ch.EPollSelectorImpl@6a6bbdf2
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000840305c40.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@14a71b33
  2> 
  2> "CloserThread" ID=33 WAITING on java.lang.Object@13df8a1b
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.Object@13df8a1b
  2> 	at java.base@11.0.16/java.lang.Object.wait(Object.java:328)
  2> 	at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2564)
  2> 
  2> "ForkJoinPool.commonPool-worker-3" ID=36 WAITING on java.util.concurrent.ForkJoinPool@525b5c3
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.ForkJoinPool@525b5c3
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
  2> 	at java.base@11.0.16/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
  2> 
  2> "searcherExecutor-13-thread-1" ID=37 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5ebcd662
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5ebcd662
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> ^^==============================================
  2> 
   >     java.lang.Exception: Test abandoned because suite timeout was reached.
   >         at __randomizedtesting.SeedInfo.seed([11BD7105C2DBC451]:0)
  2> NOTE: reproduce with: gradlew test --tests TestFieldLengthFeature.testRankingDynamicField -Dtests.seed=11BD7105C2DBC451 -Dtests.locale=chr-US -Dtests.timezone=Arctic/Longyearbyen -Dtests.asserts=true -Dtests.file.encoding=UTF-8
  2> ᎫᏰ 23, 2023 1:16:54 ᏒᎯᏱᎢᏗᏢ com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> WARNING: Will linger awaiting termination of 16 leaked thread(s).
  2> ᎫᏰ 23, 2023 1:16:55 ᏒᎯᏱᎢᏗᏢ com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> SEVERE: 16 threads leaked from SUITE scope at org.apache.solr.ltr.feature.TestFieldLengthFeature: 
  2>    1) Thread[id=18, name=qtp1784879780-18-acceptor-0@4bc46881-ServerConnector@47e867d{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:34395}, state=RUNNABLE, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
  2>         at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
  2>         at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    2) Thread[id=30, name=h2sc-8-thread-1, state=RUNNABLE, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000840305c40.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    3) Thread[id=23, name=qtp1784879780-23, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    4) Thread[id=20, name=qtp1784879780-20, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    5) Thread[id=24, name=qtp1784879780-24, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    6) Thread[id=17, name=qtp1784879780-17, state=RUNNABLE, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000840305c40.run(Unknown Source)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    7) Thread[id=25, name=Session-HouseKeeper-41ed08f0-1, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
  2>         at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    8) Thread[id=31, name=h2sc-8-thread-2, state=RUNNABLE, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000840305c40.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    9) Thread[id=22, name=qtp1784879780-22, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   10) Thread[id=15, name=TEST-TestFieldLengthFeature.testRankingDynamicField-seed#[11BD7105C2DBC451], state=WAITING, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
  2>         at java.base@11.0.16/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
  2>         at app//org.apache.solr.servlet.CoreContainerProvider$ContextInitializationKey.waitForReadyService(CoreContainerProvider.java:525)
  2>         at app//org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:564)
  2>         at app//org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
  2>         at app//org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992)
  2>         at app//org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480)
  2>         at app//org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427)
  2>         at app//org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253)
  2>         at app//org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525)
  2>         at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126)
  2>         at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86)
  2>         at app//org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57)
  2>         at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181)
  2>         at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:113)
  2>         at app//org.apache.solr.ltr.feature.TestFieldLengthFeature.before(TestFieldLengthFeature.java:31)
  2>         at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2>         at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2>         at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2>         at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:980)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  2>         at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2>         at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  2>         at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2>         at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$$Lambda$177/0x00000008401edc40.run(Unknown Source)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   11) Thread[id=19, name=qtp1784879780-19, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   12) Thread[id=33, name=CloserThread, state=WAITING, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2>         at java.base@11.0.16/java.lang.Object.wait(Object.java:328)
  2>         at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2564)
  2>   13) Thread[id=37, name=searcherExecutor-13-thread-1, state=WAITING, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2>         at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   14) Thread[id=21, name=qtp1784879780-21, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   15) Thread[id=26, name=httpShardExecutor-6-thread-1, state=RUNNABLE, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000840305c40.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   16) Thread[id=27, name=httpShardExecutor-6-thread-2, state=RUNNABLE, group=TGRP-TestFieldLengthFeature]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x0000000840305c40.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> ᎫᏰ 23, 2023 1:16:55 ᏒᎯᏱᎢᏗᏢ com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFO: Starting to interrupt leaked threads:
  2>    1) Thread[id=18, name=qtp1784879780-18-acceptor-0@4bc46881-ServerConnector@47e867d{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:34395}, state=RUNNABLE, group=TGRP-TestFieldLengthFeature]
  2>    2) Thread[id=30, name=h2sc-8-thread-1, state=RUNNABLE, group=TGRP-TestFieldLengthFeature]
  2>    3) Thread[id=23, name=qtp1784879780-23, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>    4) Thread[id=20, name=qtp1784879780-20, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>    5) Thread[id=24, name=qtp1784879780-24, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>    6) Thread[id=17, name=qtp1784879780-17, state=RUNNABLE, group=TGRP-TestFieldLengthFeature]
  2>    7) Thread[id=25, name=Session-HouseKeeper-41ed08f0-1, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>    8) Thread[id=31, name=h2sc-8-thread-2, state=RUNNABLE, group=TGRP-TestFieldLengthFeature]
  2>    9) Thread[id=22, name=qtp1784879780-22, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>   10) Thread[id=15, name=TEST-TestFieldLengthFeature.testRankingDynamicField-seed#[11BD7105C2DBC451], state=WAITING, group=TGRP-TestFieldLengthFeature]
  2>   11) Thread[id=19, name=qtp1784879780-19, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>   12) Thread[id=33, name=CloserThread, state=WAITING, group=TGRP-TestFieldLengthFeature]
  2>   13) Thread[id=37, name=searcherExecutor-13-thread-1, state=WAITING, group=TGRP-TestFieldLengthFeature]
  2>   14) Thread[id=21, name=qtp1784879780-21, state=TIMED_WAITING, group=TGRP-TestFieldLengthFeature]
  2>   15) Thread[id=26, name=httpShardExecutor-6-thread-1, state=RUNNABLE, group=TGRP-TestFieldLengthFeature]
  2>   16) Thread[id=27, name=httpShardExecutor-6-thread-2, state=RUNNABLE, group=TGRP-TestFieldLengthFeature]
  1> 11:16:55.782 [TEST-TestFieldLengthFeature.testRankingDynamicField-seed#[11BD7105C2DBC451]] ERROR org.apache.solr.servlet.SolrDispatchFilter - Could not start Dispatch Filter.
  1> org.apache.solr.common.SolrException: Interrupted while obtaining reference to CoreService
  1> 	at org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:571) ~[solr-core-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147) [solr-core-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253) [jetty-util-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94) [jetty-util-10.0.15.jar:10.0.15]
  1> 	at org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181) [test/:?]
  1> 	at org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:113) [test/:?]
  1> 	at org.apache.solr.ltr.feature.TestFieldLengthFeature.before(TestFieldLengthFeature.java:31) [test/:?]
  1> 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
  1> 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
  1> 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
  1> 	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:980) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at java.lang.Thread.run(Thread.java:829) [?:?]
  2> NOTE: leaving temporary files on disk at: /tmp/src/solr/solr/modules/ltr/build/tmp/tests-tmp/solr.ltr.feature.TestFieldLengthFeature_11BD7105C2DBC451-001
  2> Jul 23, 2023 11:16:56 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFO: All leaked threads terminated.
   >     java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   >         at __randomizedtesting.SeedInfo.seed([11BD7105C2DBC451]:0)
  2> NOTE: test params are: codec=CheapBastard, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=chr-US, timezone=Arctic/Longyearbyen
  2> NOTE: Linux 5.4.0-100-generic amd64/Oracle Corporation 11.0.16 (64-bit)/cpus=1,threads=1,free=253474048,total=324534272
  2> NOTE: All tests run in this JVM: [TestFieldLengthFeature]
  2> NOTE: reproduce with: gradlew test --tests TestFieldLengthFeature -Dtests.seed=11BD7105C2DBC451 -Dtests.locale=chr-US -Dtests.timezone=Arctic/Longyearbyen -Dtests.asserts=true -Dtests.file.encoding=UTF-8

:solr:modules:ltr:test (FAILURE): 183 test(s), 2 failure(s)

183 tests completed, 2 failed

> Task :solr:modules:ltr:test FAILED
> Task :solr:modules:ltr:wipeTaskTemp

ERROR: The following test(s) have failed:
  - org.apache.solr.ltr.feature.TestFieldLengthFeature.testRankingDynamicField (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.feature.TestFieldLengthFeature.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.feature.TestFieldLengthFeature.testRankingDynamicField" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=11BD7105C2DBC451 -Ptests.file.encoding=ISO-8859-1

  - org.apache.solr.ltr.feature.TestFieldLengthFeature.classMethod (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.feature.TestFieldLengthFeature.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.feature.TestFieldLengthFeature.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=11BD7105C2DBC451 -Ptests.file.encoding=ISO-8859-1


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:modules:ltr:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/modules/ltr/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 2h 2m 46s
642 actionable tasks: 626 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 2h3m10.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4953 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4953/

1 tests failed.
FAILED:  org.apache.solr.cli.TestSolrCLIRunExample.testInteractiveSolrCloudExample

Error Message:
java.lang.AssertionError: After running Solr cloud example, test collection 'testCloudExamplePrompt' not found in Solr at: http://localhost:35589/solr; tool output: 
Welcome to the SolrCloud example!

This interactive session will help you launch a SolrCloud cluster on your local workstation.
To begin, how many Solr nodes would you like to run in your local cluster? (specify 1-4 nodes) [2]: 
Ok, let's start up 1 Solr nodes for your example SolrCloud cluster.
Please enter the port for node1 [8983]: 
Creating Solr home directory /tmp/src/solr/solr/core/build/tmp/tests-tmp/solr.cli.TestSolrCLIRunExample_CAE25B41FBB43586-001/tempDir-001/cloud/node1/solr

Starting up Solr on port 35589 using command:
"/tmp/src/solr/solr/bin/solr" start -cloud -p 35589 -s "/tmp/src/solr/solr/core/build/tmp/tests-tmp/solr.cli.TestSolrCLIRunExample_CAE25B41FBB43586-001/tempDir-001/cloud/node1/solr"



Stack Trace:
java.lang.AssertionError: After running Solr cloud example, test collection 'testCloudExamplePrompt' not found in Solr at: http://localhost:35589/solr; tool output: 
Welcome to the SolrCloud example!

This interactive session will help you launch a SolrCloud cluster on your local workstation.
To begin, how many Solr nodes would you like to run in your local cluster? (specify 1-4 nodes) [2]: 
Ok, let's start up 1 Solr nodes for your example SolrCloud cluster.
Please enter the port for node1 [8983]: 
Creating Solr home directory /tmp/src/solr/solr/core/build/tmp/tests-tmp/solr.cli.TestSolrCLIRunExample_CAE25B41FBB43586-001/tempDir-001/cloud/node1/solr

Starting up Solr on port 35589 using command:
"/tmp/src/solr/solr/bin/solr" start -cloud -p 35589 -s "/tmp/src/solr/solr/core/build/tmp/tests-tmp/solr.cli.TestSolrCLIRunExample_CAE25B41FBB43586-001/tempDir-001/cloud/node1/solr"


	at __randomizedtesting.SeedInfo.seed([CAE25B41FBB43586:1193BB8BCCC1F0E0]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.apache.solr.cli.TestSolrCLIRunExample.testInteractiveSolrCloudExample(TestSolrCLIRunExample.java:507)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)




Build Log:
[...truncated 1259 lines...]
  2> ERROR: java.lang.Exception: Error starting up MiniSolrCloudCluster
  2> 
  2> 7009 INFO  (TEST-TestSolrCLIRunExample.testInteractiveSolrCloudExample-seed#[CAE25B41FBB43586]) [] o.a.s.SolrTestCaseJ4 ###Ending testInteractiveSolrCloudExample
   >     java.lang.AssertionError: After running Solr cloud example, test collection 'testCloudExamplePrompt' not found in Solr at: http://localhost:35589/solr; tool output: 
   >     Welcome to the SolrCloud example!
   > 
   >     This interactive session will help you launch a SolrCloud cluster on your local workstation.
   >     To begin, how many Solr nodes would you like to run in your local cluster? (specify 1-4 nodes) [2]: 
   >     Ok, let's start up 1 Solr nodes for your example SolrCloud cluster.
   >     Please enter the port for node1 [8983]: 
   >     Creating Solr home directory /tmp/src/solr/solr/core/build/tmp/tests-tmp/solr.cli.TestSolrCLIRunExample_CAE25B41FBB43586-001/tempDir-001/cloud/node1/solr
   > 
   >     Starting up Solr on port 35589 using command:
   >     "/tmp/src/solr/solr/bin/solr" start -cloud -p 35589 -s "/tmp/src/solr/solr/core/build/tmp/tests-tmp/solr.cli.TestSolrCLIRunExample_CAE25B41FBB43586-001/tempDir-001/cloud/node1/solr"
   >         at __randomizedtesting.SeedInfo.seed([CAE25B41FBB43586:1193BB8BCCC1F0E0]:0)
   >         at org.junit.Assert.fail(Assert.java:89)
   >         at org.apache.solr.cli.TestSolrCLIRunExample.testInteractiveSolrCloudExample(TestSolrCLIRunExample.java:507)
   >         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   >         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   >         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   >         at java.base/java.lang.reflect.Method.invoke(Method.java:566)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
   >         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
   >         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
   >         at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
   >         at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
   >         at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
   >         at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
   >         at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
   >         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
   >         at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
   >         at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
   >         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
   >         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
   >         at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
   >         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   >         at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   >         at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
   >         at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
   >         at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
   >         at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
   >         at org.junit.rules.RunRules.evaluate(RunRules.java:20)
   >         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
   >         at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
   >         at java.base/java.lang.Thread.run(Thread.java:829)
  2> NOTE: reproduce with: gradlew test --tests TestSolrCLIRunExample.testInteractiveSolrCloudExample -Dtests.seed=CAE25B41FBB43586 -Dtests.locale=ewo-CM -Dtests.timezone=US/Central -Dtests.asserts=true -Dtests.file.encoding=UTF-8
  2> 7048 INFO  (TEST-TestSolrCLIRunExample.testFailExecuteScript-seed#[CAE25B41FBB43586]) [] o.a.s.SolrTestCaseJ4 ###Starting testFailExecuteScript
  2> 
  2> ERROR: access denied ("java.io.FilePermission" "/tmp/src/solr/solr/core/build/tmp/tests-tmp/solr.cli.TestSolrCLIRunExample_CAE25B41FBB43586-001/tempDir-003/failExecuteScript" "execute")
  2> 
  2> 7077 INFO  (TEST-TestSolrCLIRunExample.testFailExecuteScript-seed#[CAE25B41FBB43586]) [] o.a.s.SolrTestCaseJ4 ###Ending testFailExecuteScript
  2> 7084 INFO  (SUITE-TestSolrCLIRunExample-seed#[CAE25B41FBB43586]-worker) [] o.a.s.u.ErrorLogMuter Closing ErrorLogMuter-regex-1 after mutting 0 log messages
  2> 7085 INFO  (SUITE-TestSolrCLIRunExample-seed#[CAE25B41FBB43586]-worker) [] o.a.s.u.ErrorLogMuter Creating ErrorLogMuter-regex-2 for ERROR logs matching regex: ignore_exception
  2> NOTE: leaving temporary files on disk at: /tmp/src/solr/solr/core/build/tmp/tests-tmp/solr.cli.TestSolrCLIRunExample_CAE25B41FBB43586-001
  2> NOTE: test params are: codec=Asserting(Lucene95): {}, docValues:{}, maxPointsInLeafNode=971, maxMBSortInHeap=7.993559820700737, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=ewo-CM, timezone=US/Central
  2> NOTE: Linux 5.4.0-100-generic amd64/Oracle Corporation 11.0.16 (64-bit)/cpus=1,threads=1,free=230509400,total=324534272
  2> NOTE: All tests run in this JVM: [TestSolrCLIRunExample]


> Task :solr:modules:hdfs:test
:solr:modules:hdfs:test (SUCCESS): 102 test(s), 52 skipped

> Task :solr:modules:hdfs:wipeTaskTemp
> Task :solr:modules:hdfs:check

> Task :solr:core:test
WARNING: Test org.apache.solr.schema.TestCloudSchemaless wrote 12,035,386 bytes of output.
WARNING: Test org.apache.solr.search.facet.TestCloudJSONFacetSKG wrote 19,770,734 bytes of output.
WARNING: Test org.apache.solr.search.facet.TestCloudJSONFacetSKGEquiv wrote 15,337,430 bytes of output.
:solr:core:test (FAILURE): 4847 test(s), 1 failure(s), 213 skipped

4847 tests completed, 1 failed, 213 skipped

> Task :solr:core:test FAILED
> Task :solr:core:wipeTaskTemp

> Task :solr:packaging:integrationTests
ok 4 create for non cloud mode in 15677ms
ok 5 create for cloud mode in 16657ms
ok 6 create collection in 7797ms
ok 7 reject d option with invalid config dir in 1275ms
ok 8 accept d option with builtin config in 5796ms
ok 9 accept d option with explicit path to config in 6309ms
ok 10 accept n option as config name in 6139ms
ok 11 allow config reuse when n option specifies same config in 10499ms
ok 12 create multisharded collections when s provided in 5844ms
ok 13 create replicated collections when rf provided in 6921ms
ok 14 can delete collections in 9927ms
ok 15 collection delete also deletes zk config in 8325ms
ok 16 deletes accompanying zk config with nondefault name in 8282ms
ok 17 deleteConfig option can opt to leave config in zk in 7881ms
ok 18 SOLR-16755 test works with noprompt in 32180ms
ok 19 Check export command in 28987ms
ok 20 export fails on non cloud mode in 17087ms
ok 21 healthcheck on cloud solr in 21652ms
ok 22 start help flag prints help in 150ms
ok 23 stop help flag prints help in 144ms
ok 24 restart help flag prints help in 150ms
ok 25 status help flag prints help in 48ms # skip Currently the status -help flag doesn't return nice help text!
ok 26 healthcheck help flag prints help in 142ms
ok 27 create help flag prints help in 142ms
ok 28 createcore help flag prints help in 144ms
ok 29 createcollection help flag prints help in 145ms
ok 30 delete help flag prints help in 144ms
ok 31 version help flag prints help in 47ms # skip Currently the version -help flag doesn't return nice help text!
ok 32 zk help flag prints help in 147ms
ok 33 auth help flag prints help in 145ms
ok 34 assert help flag prints help in 47ms # skip Currently the assert -help flag doesn't return nice help text!
ok 35 SQL Module in 23701ms
ok 36 Hadoop-Auth Module: KerberosPlugin Classloading in 26078ms
ok 37 icu collation in analysis-extras module in 19264ms
ok 38 package detects no running solr in 229ms
ok 39 lifecycle of package in 16455ms
ok 40 deploying and undeploying of packages in 20384ms
not ok 41 Affinity placement plugin using sysprop in 18900ms
# (from function `collection_exists' in file test/bats_helper.bash, line 93,
#  in test file test/test_placement_plugin.bats, line 36)
#   `collection_exists COLL_NAME' failed
# WARN  - 2023-07-23 07:41:49.427; org.apache.solr.common.cloud.SolrZkClient; Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
# WARN  - 2023-07-23 07:41:49.485; org.apache.solr.common.cloud.SolrZkClient; Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
# Please find the SOLR_HOME snapshot for failed test #1 at: /tmp/src/solr/solr/packaging/build/test-output/failure-snapshots/41-1
# WARN  - 2023-07-23 07:41:50.556; org.apache.solr.common.cloud.SolrZkClient; Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
# WARN  - 2023-07-23 07:41:50.616; org.apache.solr.common.cloud.SolrZkClient; Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
# Last output:
# WARNING: Using _default configset with data driven schema functionality. NOT RECOMMENDED for production use.
#          To turn off: bin/solr config -c COLL_NAME -p 8983 -action set-user-property -property update.autoCreateFields -value false
# WARN  - 2023-07-23 07:41:47.699; org.apache.solr.common.cloud.SolrZkClient; Using default ZkCredentialsInjector. ZkCredentialsInjector is not secure, it creates an empty list of credentials which leads to 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
# WARN  - 2023-07-23 07:41:47.743; org.apache.solr.common.cloud.SolrZkClient; Using default ZkACLProvider. DefaultZkACLProvider is not secure, it creates 'OPEN_ACL_UNSAFE' ACLs to Zookeeper nodes
#
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22549ms
ok 43 post solr log into solr in 9130ms
ok 44 start solr with ssl in 20239ms
ok 45 start solr with ssl and auth in 22977ms
ok 46 SOLR-11740 check 'solr stop' connection in 28185ms
ok 47 status detects locally running solr in 14471ms
ok 48 status does not expose cli parameters to end user in 218ms
ok 49 listing out files in 2214ms
ok 50 copying files around in 6708ms
ok 51 upconfig in 2670ms

> Task :solr:packaging:integrationTests FAILED

ERROR: The following test(s) have failed:
  - org.apache.solr.cli.TestSolrCLIRunExample.testInteractiveSolrCloudExample (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cli.TestSolrCLIRunExample.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cli.TestSolrCLIRunExample.testInteractiveSolrCloudExample" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=CAE25B41FBB43586 -Ptests.file.encoding=ISO-8859-1


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:core:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/core/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 15m 59s
625 actionable tasks: 609 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m24s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4952 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4952/

All tests passed

Build Log:
[...truncated 1257 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22751ms
ok 43 post solr log into solr in 9559ms
ok 44 start solr with ssl in 20288ms
ok 45 start solr with ssl and auth in 22516ms
ok 46 SOLR-11740 check 'solr stop' connection in 28485ms
ok 47 status detects locally running solr in 14737ms
ok 48 status does not expose cli parameters to end user in 218ms
ok 49 listing out files in 2197ms
ok 50 copying files around in 6613ms
ok 51 upconfig in 2662ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 5s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m30s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4951 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4951/

4 tests failed.
FAILED:  org.apache.solr.ltr.feature.TestNoMatchSolrFeature.test2NoMatch1YesMatchFeatureReturnsFvWith1FeatureAndDocScoreScaledByModel

Error Message:
java.lang.Exception: Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([9C25EB8474ACF490]:0)


FAILED:  org.apache.solr.ltr.feature.TestNoMatchSolrFeature.classMethod

Error Message:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([9C25EB8474ACF490]:0)


FAILED:  org.apache.solr.ltr.model.TestDefaultWrapperModel.testLoadNestedWrapperModel

Error Message:
java.lang.Exception: Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([9C25EB8474ACF490]:0)


FAILED:  org.apache.solr.ltr.model.TestDefaultWrapperModel.classMethod

Error Message:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([9C25EB8474ACF490]:0)




Build Log:
[...truncated 1243 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 23146ms
ok 43 post solr log into solr in 9203ms
ok 44 start solr with ssl in 20526ms
ok 45 start solr with ssl and auth in 22642ms
ok 46 SOLR-11740 check 'solr stop' connection in 28195ms
ok 47 status detects locally running solr in 14560ms
ok 48 status does not expose cli parameters to end user in 217ms
ok 49 listing out files in 2242ms
ok 50 copying files around in 6703ms
ok 51 upconfig in 2645ms

> Task :solr:packaging:integrationTests FAILED

> Task :solr:modules:ltr:test

org.apache.solr.ltr.feature.TestNoMatchSolrFeature > test2NoMatch1YesMatchFeatureReturnsFvWith1FeatureAndDocScoreScaledByModel FAILED
    java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([9C25EB8474ACF490]:0)

org.apache.solr.ltr.feature.TestNoMatchSolrFeature > classMethod FAILED
    java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([9C25EB8474ACF490]:0)

org.apache.solr.ltr.feature.TestNoMatchSolrFeature > test suite's output saved to /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.feature.TestNoMatchSolrFeature.txt, copied below:
  1> 23:45:01.957 [TEST-TestNoMatchSolrFeature.test2NoMatch1YesMatchFeatureReturnsFvWith1FeatureAndDocScoreScaledByModel-seed#[9C25EB8474ACF490]] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
  2> jul. 23, 2023 11:45:00 A. M. com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
  2> ADVERTENCIA: Suite execution timed out: org.apache.solr.ltr.feature.TestNoMatchSolrFeature
  2> ==== jstack at approximately timeout time ====
  2> "Test worker" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$2@44093b5e
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$2@44093b5e
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1300)
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1375)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:647)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:504)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:108)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:57)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:39)
  2> 	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
  2> 	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:52)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
  2> 	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
  2> 	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
  2> 	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
  2> 	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
  2> 
  2> "Reference Handler" ID=2 RUNNABLE
  2> 	at java.base@11.0.16/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
  2> 	at java.base@11.0.16/java.lang.ref.Reference.processPendingReferences(Reference.java:241)
  2> 	at java.base@11.0.16/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)
  2> 
  2> "Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@1938b377
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.ref.ReferenceQueue$Lock@1938b377
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
  2> 	at java.base@11.0.16/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)
  2> 
  2> "Signal Dispatcher" ID=4 RUNNABLE
  2> 
  2> "Common-Cleaner" ID=9 TIMED_WAITING on java.lang.ref.ReferenceQueue$Lock@699d93d1
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- timed waiting on java.lang.ref.ReferenceQueue$Lock@699d93d1
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	at java.base@11.0.16/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:161)
  2> 
  2> "/127.0.0.1:37632 to /127.0.0.1:42531 workers" ID=11 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@686e7fc3
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@686e7fc3
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:403)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@42e99e4a
  2> 
  2> "/127.0.0.1:37632 to /127.0.0.1:42531 workers Thread 2" ID=12 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@52d8e2b1
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@52d8e2b1
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionDispatch.run(MessageHub.java:322)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@42a48628
  2> 
  2> "/127.0.0.1:37632 to /127.0.0.1:42531 workers Thread 3" ID=13 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@ea74096
  2> 	- locked sun.nio.ch.EPollSelectorImpl@ba14988
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at org.gradle.internal.remote.internal.inet.SocketConnection$SocketInputStream.read(SocketConnection.java:185)
  2> 	at com.esotericsoftware.kryo.io.Input.fill(Input.java:146)
  2> 	at com.esotericsoftware.kryo.io.Input.require(Input.java:178)
  2> 	at com.esotericsoftware.kryo.io.Input.readByte(Input.java:295)
  2> 	at org.gradle.internal.serialize.kryo.KryoBackedDecoder.readByte(KryoBackedDecoder.java:88)
  2> 	at org.gradle.internal.remote.internal.hub.InterHubMessageSerializer$MessageReader.read(InterHubMessageSerializer.java:64)
  2> 	at org.gradle.internal.remote.internal.hub.InterHubMessageSerializer$MessageReader.read(InterHubMessageSerializer.java:52)
  2> 	at org.gradle.internal.remote.internal.inet.SocketConnection.receive(SocketConnection.java:81)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionReceive.run(MessageHub.java:270)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@74f0ea28
  2> 
  2> "SUITE-TestNoMatchSolrFeature-seed#[9C25EB8474ACF490]" ID=14 RUNNABLE
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpThreads0(Native Method)
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:521)
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:509)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:708)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:59)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:437)
  2> 	- locked java.lang.Object@10aaaf58
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:716)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:637)
  2> 
  2> "TEST-TestNoMatchSolrFeature.test2NoMatch1YesMatchFeatureReturnsFvWith1FeatureAndDocScoreScaledByModel-seed#[9C25EB8474ACF490]" ID=15 WAITING on java.util.concurrent.CountDownLatch$Sync@7b1b8a38
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.CountDownLatch$Sync@7b1b8a38
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
  2> 	at java.base@11.0.16/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
  2> 	at app//org.apache.solr.servlet.CoreContainerProvider$ContextInitializationKey.waitForReadyService(CoreContainerProvider.java:525)
  2> 	at app//org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:564)
  2> 	at app//org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
  2> 	at app//org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992)
  2> 	at app//org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427)
  2> 	- locked org.apache.solr.embedded.JettySolrRunner$1@5f20a96
  2> 	at app//org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253)
  2> 	at app//org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554)
  2> 	- locked org.apache.solr.embedded.JettySolrRunner@31e5494e
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525)
  2> 	at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126)
  2> 	at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86)
  2> 	at app//org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57)
  2> 	at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181)
  2> 	at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:113)
  2> 	at app//org.apache.solr.ltr.feature.TestNoMatchSolrFeature.before(TestNoMatchSolrFeature.java:35)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:980)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2> 	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$$Lambda$177/0x00000008401edc40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@3089abf3
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@751c02d5
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@60af005a
  2> 
  2> "qtp1951306895-17" ID=17 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@630709b
  2> 	- locked sun.nio.ch.EPollSelectorImpl@132fd022
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034fc40.run(Unknown Source)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1951306895-18-acceptor-0@5d4eef47-ServerConnector@48a55b3b{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:45173}" ID=18 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
  2> 	at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
  2> 	at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@b1aafcd
  2> 
  2> "qtp1951306895-19" ID=19 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c77bedc
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c77bedc
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1951306895-20" ID=20 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c77bedc
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c77bedc
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1951306895-21" ID=21 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c77bedc
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c77bedc
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1951306895-22" ID=22 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c77bedc
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c77bedc
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1951306895-23" ID=23 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c77bedc
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c77bedc
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1951306895-24" ID=24 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c77bedc
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2c77bedc
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "Session-HouseKeeper-1b0f8b7d-1" ID=25 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2946b761
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@2946b761
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
  2> 	at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "httpShardExecutor-6-thread-1" ID=26 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@4485dcab
  2> 	- locked sun.nio.ch.EPollSelectorImpl@1abf7731
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034fc40.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x000000084044d040.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@72ba4a95
  2> 
  2> "httpShardExecutor-6-thread-2" ID=27 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@b2fabe4
  2> 	- locked sun.nio.ch.EPollSelectorImpl@17cee85f
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034fc40.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x000000084044d040.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@61bffcf4
  2> 
  2> "Connection evictor" ID=28 TIMED_WAITING
  2> 	at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
  2> 	at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "Connection evictor" ID=29 TIMED_WAITING
  2> 	at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
  2> 	at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "h2sc-8-thread-1" ID=30 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@7dc40e89
  2> 	- locked sun.nio.ch.EPollSelectorImpl@1c23f8ff
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034fc40.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x000000084044d040.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@3c1f1775
  2> 
  2> "h2sc-8-thread-2" ID=31 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@6a251014
  2> 	- locked sun.nio.ch.EPollSelectorImpl@ac9911f
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034fc40.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x000000084044d040.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@5807edc4
  2> 
  2> "CloserThread" ID=33 WAITING on java.lang.Object@57247689
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.Object@57247689
  2> 	at java.base@11.0.16/java.lang.Object.wait(Object.java:328)
  2> 	at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2564)
  2> 
  2> "ForkJoinPool.commonPool-worker-3" ID=36 WAITING on java.util.concurrent.ForkJoinPool@54438892
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.ForkJoinPool@54438892
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
  2> 	at java.base@11.0.16/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
  2> 
  2> "searcherExecutor-13-thread-1" ID=37 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@162497be
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@162497be
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> ^^==============================================
  2> 
   >     java.lang.Exception: Test abandoned because suite timeout was reached.
   >         at __randomizedtesting.SeedInfo.seed([9C25EB8474ACF490]:0)
  2> NOTE: reproduce with: gradlew test --tests TestNoMatchSolrFeature.test2NoMatch1YesMatchFeatureReturnsFvWith1FeatureAndDocScoreScaledByModel -Dtests.seed=9C25EB8474ACF490 -Dtests.locale=es-US -Dtests.timezone=Australia/Tasmania -Dtests.asserts=true -Dtests.file.encoding=UTF-8
  2> jul. 23, 2023 11:45:00 A. M. com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> ADVERTENCIA: Will linger awaiting termination of 16 leaked thread(s).
  2> jul. 23, 2023 11:45:01 A. M. com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> GRAVE: 16 threads leaked from SUITE scope at org.apache.solr.ltr.feature.TestNoMatchSolrFeature: 
  2>    1) Thread[id=25, name=Session-HouseKeeper-1b0f8b7d-1, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
  2>         at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    2) Thread[id=31, name=h2sc-8-thread-2, state=RUNNABLE, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034fc40.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x000000084044d040.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    3) Thread[id=23, name=qtp1951306895-23, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    4) Thread[id=30, name=h2sc-8-thread-1, state=RUNNABLE, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034fc40.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x000000084044d040.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    5) Thread[id=37, name=searcherExecutor-13-thread-1, state=WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2>         at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    6) Thread[id=22, name=qtp1951306895-22, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    7) Thread[id=15, name=TEST-TestNoMatchSolrFeature.test2NoMatch1YesMatchFeatureReturnsFvWith1FeatureAndDocScoreScaledByModel-seed#[9C25EB8474ACF490], state=WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
  2>         at java.base@11.0.16/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
  2>         at app//org.apache.solr.servlet.CoreContainerProvider$ContextInitializationKey.waitForReadyService(CoreContainerProvider.java:525)
  2>         at app//org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:564)
  2>         at app//org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
  2>         at app//org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992)
  2>         at app//org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480)
  2>         at app//org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427)
  2>         at app//org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253)
  2>         at app//org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525)
  2>         at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126)
  2>         at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86)
  2>         at app//org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57)
  2>         at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181)
  2>         at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:113)
  2>         at app//org.apache.solr.ltr.feature.TestNoMatchSolrFeature.before(TestNoMatchSolrFeature.java:35)
  2>         at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2>         at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2>         at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2>         at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:980)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  2>         at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2>         at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  2>         at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2>         at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$$Lambda$177/0x00000008401edc40.run(Unknown Source)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    8) Thread[id=33, name=CloserThread, state=WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2>         at java.base@11.0.16/java.lang.Object.wait(Object.java:328)
  2>         at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2564)
  2>    9) Thread[id=27, name=httpShardExecutor-6-thread-2, state=RUNNABLE, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034fc40.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x000000084044d040.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   10) Thread[id=26, name=httpShardExecutor-6-thread-1, state=RUNNABLE, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034fc40.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$356/0x000000084044d040.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   11) Thread[id=19, name=qtp1951306895-19, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   12) Thread[id=21, name=qtp1951306895-21, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   13) Thread[id=24, name=qtp1951306895-24, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   14) Thread[id=17, name=qtp1951306895-17, state=RUNNABLE, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$258/0x000000084034fc40.run(Unknown Source)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   15) Thread[id=20, name=qtp1951306895-20, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   16) Thread[id=18, name=qtp1951306895-18-acceptor-0@5d4eef47-ServerConnector@48a55b3b{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:45173}, state=RUNNABLE, group=TGRP-TestNoMatchSolrFeature]
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
  2>         at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
  2>         at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> jul. 23, 2023 11:45:01 A. M. com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFORMACIÓN: Starting to interrupt leaked threads:
  2>    1) Thread[id=25, name=Session-HouseKeeper-1b0f8b7d-1, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>    2) Thread[id=31, name=h2sc-8-thread-2, state=RUNNABLE, group=TGRP-TestNoMatchSolrFeature]
  2>    3) Thread[id=23, name=qtp1951306895-23, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>    4) Thread[id=30, name=h2sc-8-thread-1, state=RUNNABLE, group=TGRP-TestNoMatchSolrFeature]
  2>    5) Thread[id=37, name=searcherExecutor-13-thread-1, state=WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>    6) Thread[id=22, name=qtp1951306895-22, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>    7) Thread[id=15, name=TEST-TestNoMatchSolrFeature.test2NoMatch1YesMatchFeatureReturnsFvWith1FeatureAndDocScoreScaledByModel-seed#[9C25EB8474ACF490], state=WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>    8) Thread[id=33, name=CloserThread, state=WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>    9) Thread[id=27, name=httpShardExecutor-6-thread-2, state=RUNNABLE, group=TGRP-TestNoMatchSolrFeature]
  2>   10) Thread[id=26, name=httpShardExecutor-6-thread-1, state=RUNNABLE, group=TGRP-TestNoMatchSolrFeature]
  2>   11) Thread[id=19, name=qtp1951306895-19, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>   12) Thread[id=21, name=qtp1951306895-21, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>   13) Thread[id=24, name=qtp1951306895-24, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>   14) Thread[id=17, name=qtp1951306895-17, state=RUNNABLE, group=TGRP-TestNoMatchSolrFeature]
  2>   15) Thread[id=20, name=qtp1951306895-20, state=TIMED_WAITING, group=TGRP-TestNoMatchSolrFeature]
  2>   16) Thread[id=18, name=qtp1951306895-18-acceptor-0@5d4eef47-ServerConnector@48a55b3b{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:45173}, state=RUNNABLE, group=TGRP-TestNoMatchSolrFeature]
  1> 01:45:01.324 [TEST-TestNoMatchSolrFeature.test2NoMatch1YesMatchFeatureReturnsFvWith1FeatureAndDocScoreScaledByModel-seed#[9C25EB8474ACF490]] ERROR org.apache.solr.servlet.SolrDispatchFilter - Could not start Dispatch Filter.
  1> org.apache.solr.common.SolrException: Interrupted while obtaining reference to CoreService
  1> 	at org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:571) ~[solr-core-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147) [solr-core-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253) [jetty-util-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94) [jetty-util-10.0.15.jar:10.0.15]
  1> 	at org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 9b61ba603fbcc19c1bdd5d7826f9e80faf30a02a [snapshot build, details omitted]]
  1> 	at org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181) [test/:?]
  1> 	at org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:113) [test/:?]
  1> 	at org.apache.solr.ltr.feature.TestNoMatchSolrFeature.before(TestNoMatchSolrFeature.java:35) [test/:?]
  1> 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
  1> 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
  1> 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
  1> 	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:980) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at java.lang.Thread.run(Thread.java:829) [?:?]
  2> NOTE: leaving temporary files on disk at: /tmp/src/solr/solr/modules/ltr/build/tmp/tests-tmp/solr.ltr.feature.TestNoMatchSolrFeature_9C25EB8474ACF490-001
  2> Jul 23, 2023 1:45:02 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFO: All leaked threads terminated.
   >     java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   >         at __randomizedtesting.SeedInfo.seed([9C25EB8474ACF490]:0)
  2> NOTE: test params are: codec=Asserting(Lucene95): {}, docValues:{}, maxPointsInLeafNode=134, maxMBSortInHeap=7.319360345048614, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=es-US, timezone=Australia/Tasmania
  2> NOTE: Linux 5.4.0-100-generic amd64/Oracle Corporation 11.0.16 (64-bit)/cpus=1,threads=1,free=251579776,total=324534272
  2> NOTE: All tests run in this JVM: [TestNoMatchSolrFeature]
  2> NOTE: reproduce with: gradlew test --tests TestNoMatchSolrFeature -Dtests.seed=9C25EB8474ACF490 -Dtests.locale=es-US -Dtests.timezone=Australia/Tasmania -Dtests.asserts=true -Dtests.file.encoding=UTF-8


org.apache.solr.ltr.model.TestDefaultWrapperModel > testLoadNestedWrapperModel FAILED
    java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([9C25EB8474ACF490]:0)

org.apache.solr.ltr.model.TestDefaultWrapperModel > classMethod FAILED
    java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([9C25EB8474ACF490]:0)

org.apache.solr.ltr.model.TestDefaultWrapperModel > test suite's output saved to /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.model.TestDefaultWrapperModel.txt, copied below:
  1> 23:45:10.809 [TEST-TestDefaultWrapperModel.testLoadNestedWrapperModel-seed#[9C25EB8474ACF490]] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
  2> NJR 23, 2023 2:45:09 RŨ com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
  2> WARNING: Suite execution timed out: org.apache.solr.ltr.model.TestDefaultWrapperModel
  2> ==== jstack at approximately timeout time ====
  2> "Test worker" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$2@3fb55b3e
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$2@3fb55b3e
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1300)
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1375)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:647)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:504)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:108)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:57)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:39)
  2> 	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
  2> 	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:52)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
  2> 	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
  2> 	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
  2> 	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
  2> 	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
  2> 
  2> "Reference Handler" ID=2 RUNNABLE
  2> 	at java.base@11.0.16/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
  2> 	at java.base@11.0.16/java.lang.ref.Reference.processPendingReferences(Reference.java:241)
  2> 	at java.base@11.0.16/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)
  2> 
  2> "Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@49c1a346
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.ref.ReferenceQueue$Lock@49c1a346
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
  2> 	at java.base@11.0.16/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)
  2> 
  2> "Signal Dispatcher" ID=4 RUNNABLE
  2> 
  2> "Common-Cleaner" ID=9 TIMED_WAITING on java.lang.ref.ReferenceQueue$Lock@434de44e
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- timed waiting on java.lang.ref.ReferenceQueue$Lock@434de44e
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	at java.base@11.0.16/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:161)
  2> 
  2> "/127.0.0.1:49988 to /127.0.0.1:33995 workers" ID=11 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1313570
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1313570
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at 

[...truncated too long message...]

 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at java.lang.Thread.run(Thread.java:829) [?:?]
  2> NOTE: leaving temporary files on disk at: /tmp/src/solr/solr/modules/ltr/build/tmp/tests-tmp/solr.ltr.model.TestDefaultWrapperModel_9C25EB8474ACF490-001
  2> Jul 23, 2023 1:45:11 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFO: All leaked threads terminated.
   >     java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   >         at __randomizedtesting.SeedInfo.seed([9C25EB8474ACF490]:0)
  2> NOTE: test params are: codec=Lucene95, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=mer-KE, timezone=Atlantic/Madeira
  2> NOTE: Linux 5.4.0-100-generic amd64/Oracle Corporation 11.0.16 (64-bit)/cpus=1,threads=1,free=254577272,total=324534272
  2> NOTE: All tests run in this JVM: [TestDefaultWrapperModel]
  2> NOTE: reproduce with: gradlew test --tests TestDefaultWrapperModel -Dtests.seed=9C25EB8474ACF490 -Dtests.locale=mer-KE -Dtests.timezone=Atlantic/Madeira -Dtests.asserts=true -Dtests.file.encoding=UTF-8

:solr:modules:ltr:test (FAILURE): 182 test(s), 4 failure(s)

182 tests completed, 4 failed

> Task :solr:modules:ltr:test FAILED
> Task :solr:modules:ltr:wipeTaskTemp

ERROR: The following test(s) have failed:
  - org.apache.solr.ltr.feature.TestNoMatchSolrFeature.test2NoMatch1YesMatchFeatureReturnsFvWith1FeatureAndDocScoreScaledByModel (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.feature.TestNoMatchSolrFeature.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.feature.TestNoMatchSolrFeature.test2NoMatch1YesMatchFeatureReturnsFvWith1FeatureAndDocScoreScaledByModel" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=9C25EB8474ACF490 -Ptests.file.encoding=ISO-8859-1

  - org.apache.solr.ltr.feature.TestNoMatchSolrFeature.classMethod (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.feature.TestNoMatchSolrFeature.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.feature.TestNoMatchSolrFeature.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=9C25EB8474ACF490 -Ptests.file.encoding=ISO-8859-1

  - org.apache.solr.ltr.model.TestDefaultWrapperModel.testLoadNestedWrapperModel (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.model.TestDefaultWrapperModel.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.model.TestDefaultWrapperModel.testLoadNestedWrapperModel" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=9C25EB8474ACF490 -Ptests.file.encoding=ISO-8859-1

  - org.apache.solr.ltr.model.TestDefaultWrapperModel.classMethod (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.model.TestDefaultWrapperModel.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.model.TestDefaultWrapperModel.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=9C25EB8474ACF490 -Ptests.file.encoding=ISO-8859-1


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:modules:ltr:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/modules/ltr/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 2h 2m 56s
642 actionable tasks: 626 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 2h3m19.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4950 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4950/

All tests passed

Build Log:
[...truncated 1251 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22368ms
ok 43 post solr log into solr in 8932ms
ok 44 start solr with ssl in 19821ms
ok 45 start solr with ssl and auth in 22794ms
ok 46 SOLR-11740 check 'solr stop' connection in 28104ms
ok 47 status detects locally running solr in 14372ms
ok 48 status does not expose cli parameters to end user in 211ms
ok 49 listing out files in 2179ms
ok 50 copying files around in 6630ms
ok 51 upconfig in 2635ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 8s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m33.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4949 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4949/

All tests passed

Build Log:
[...truncated 1263 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 23006ms
ok 43 post solr log into solr in 9090ms
ok 44 start solr with ssl in 19697ms
ok 45 start solr with ssl and auth in 22351ms
ok 46 SOLR-11740 check 'solr stop' connection in 28390ms
ok 47 status detects locally running solr in 14335ms
ok 48 status does not expose cli parameters to end user in 207ms
ok 49 listing out files in 2164ms
ok 50 copying files around in 6577ms
ok 51 upconfig in 2638ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 13m 3s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 13m27s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4948 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4948/

6 tests failed.
FAILED:  org.apache.solr.cloud.RecoveryZkTest.test

Error Message:
org.apache.solr.client.solrj.SolrServerException: Timeout occurred while waiting response from server at: http://127.0.0.1:41937/solr/recoverytest_shard1_replica_n2

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occurred while waiting response from server at: http://127.0.0.1:41937/solr/recoverytest_shard1_replica_n2
	at __randomizedtesting.SeedInfo.seed([D01445D7CE7006F0:58407A0D608C6B08]:0)
	at app//org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:755)
	at app//org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:263)
	at app//org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at app//org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:386)
	at app//org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:352)
	at app//org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1199)
	at app//org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:898)
	at app//org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:826)
	at app//org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:234)
	at app//org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:228)
	at app//org.apache.solr.cloud.AbstractRecoveryZkTestBase.test(AbstractRecoveryZkTestBase.java:124)
	at app//org.apache.solr.cloud.RecoveryZkTest.test(RecoveryZkTest.java:29)
	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.base/java.net.SocketInputStream.socketRead0(Native Method)
	at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168)
	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:603)
	... 58 more


FAILED:  org.apache.solr.cloud.RecoveryZkTest.classMethod

Error Message:
java.lang.Exception: Error shutting down MiniSolrCloudCluster

Stack Trace:
java.lang.Exception: Error shutting down MiniSolrCloudCluster
	at __randomizedtesting.SeedInfo.seed([D01445D7CE7006F0]:0)
	at org.apache.solr.cloud.MiniSolrCloudCluster.checkForExceptions(MiniSolrCloudCluster.java:836)
	at org.apache.solr.cloud.MiniSolrCloudCluster.shutdown(MiniSolrCloudCluster.java:739)
	at org.apache.solr.cloud.SolrCloudTestCase.shutdownCluster(SolrCloudTestCase.java:118)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:909)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)
	Suppressed: java.lang.RuntimeException: Timeout waiting for pool org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@34f903c1[Shutting down, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 3] to shutdown.
		at org.apache.solr.common.util.ExecutorUtil.awaitTermination(ExecutorUtil.java:130)
		at org.apache.solr.common.util.ExecutorUtil.awaitTermination(ExecutorUtil.java:117)
		at org.apache.solr.common.util.ExecutorUtil.shutdownAndAwaitTermination(ExecutorUtil.java:107)
		at org.apache.solr.core.CoreContainer.shutdown(CoreContainer.java:1339)
		at org.apache.solr.servlet.CoreContainerProvider.close(CoreContainerProvider.java:195)
		at org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStopped(JettySolrRunner.java:387)
		at org.eclipse.jetty.util.component.AbstractLifeCycle.setStopped(AbstractLifeCycle.java:286)
		at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:133)
		at org.apache.solr.embedded.JettySolrRunner.stop(JettySolrRunner.java:668)
		at org.apache.solr.cloud.MiniSolrCloudCluster.stopJettySolrRunner(MiniSolrCloudCluster.java:618)
		at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$shutdown$5(MiniSolrCloudCluster.java:731)
		at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
		at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
		at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
		at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
		... 1 more


FAILED:  org.apache.solr.cloud.TestConfigSetsAPIExclusivity.testAPIExclusivity

Error Message:
java.lang.AssertionError: Unexpected exception: org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44905/solr: Error getting config metadata expected:<0> but was:<1>

Stack Trace:
java.lang.AssertionError: Unexpected exception: org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:44905/solr: Error getting config metadata expected:<0> but was:<1>
	at __randomizedtesting.SeedInfo.seed([D01445D7CE7006F0:AD615C2C97D4BFB9]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:647)
	at org.apache.solr.cloud.TestConfigSetsAPIExclusivity.testAPIExclusivity(TestConfigSetsAPIExclusivity.java:88)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)


FAILED:  org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess

Error Message:
java.lang.AssertionError: expected:<4> but was:<5>

Stack Trace:
java.lang.AssertionError: expected:<4> but was:<5>
	at __randomizedtesting.SeedInfo.seed([D01445D7CE7006F0:1F097AD6AFF5FBA3]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:633)
	at org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess(TestCoordinatorRole.java:573)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)


FAILED:  org.apache.solr.search.TestCoordinatorRole.classMethod

Error Message:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from SUITE scope at org.apache.solr.search.TestCoordinatorRole: 
   1) Thread[id=1611, name=TestCoordinatorRole-1139-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=1624, name=TestCoordinatorRole-1139-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=1612, name=TestCoordinatorRole-1139-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=1620, name=TestCoordinatorRole-1139-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=1615, name=TestCoordinatorRole-1139-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=1616, name=TestCoordinatorRole-1139-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=1614, name=TestCoordinatorRole-1139-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=1623, name=TestCoordinatorRole-1139-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=1613, name=TestCoordinatorRole-1139-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=1617, name=TestCoordinatorRole-1139-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from SUITE scope at org.apache.solr.search.TestCoordinatorRole: 
   1) Thread[id=1611, name=TestCoordinatorRole-1139-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=1624, name=TestCoordinatorRole-1139-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=1612, name=TestCoordinatorRole-1139-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=1620, name=TestCoordinatorRole-1139-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=1615, name=TestCoordinatorRole-1139-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=1616, name=TestCoordinatorRole-1139-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=1614, name=TestCoordinatorRole-1139-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=1623, name=TestCoordinatorRole-1139-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=1613, name=TestCoordinatorRole-1139-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=1617, name=TestCoordinatorRole-1139-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
	at __randomizedtesting.SeedInfo.seed([D01445D7CE7006F0]:0)


FAILED:  org.apache.solr.search.TestCoordinatorRole.classMethod

Error Message:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=1611, name=TestCoordinatorRole-1139-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=1624, name=TestCoordinatorRole-1139-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=1614, name=TestCoordinatorRole-1139-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=1612, name=TestCoordinatorRole-1139-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=1623, name=TestCoordinatorRole-1139-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=1620, name=TestCoordinatorRole-1139-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=1613, name=TestCoordinatorRole-1139-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=1615, name=TestCoordinatorRole-1139-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=1617, name=TestCoordinatorRole-1139-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=1616, name=TestCoordinatorRole-1139-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=1611, name=TestCoordinatorRole-1139-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=1624, name=TestCoordinatorRole-1139-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=1614, name=TestCoordinatorRole-1139-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=1612, name=TestCoordinatorRole-1139-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=1623, name=TestCoordinatorRole-1139-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=1620, name=TestCoordinatorRole-1139-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=1613, name=TestCoordinatorRole-1139-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=1615, name=TestCoordinatorRole-1139-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=1617, name=TestCoordinatorRole-1139-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=1616, name=TestCoordinatorRole-1139-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
	at __randomizedtesting.SeedInfo.seed([D01445D7CE7006F0]:0)




Build Log:
[...truncated 12568 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22806ms
ok 43 post solr log into solr in 9156ms
ok 44 start solr with ssl in 19882ms
ok 45 start solr with ssl and auth in 22554ms
ok 46 SOLR-11740 check 'solr stop' connection in 28348ms
ok 47 status detects locally running solr in 14366ms
ok 48 status does not expose cli parameters to end user in 209ms
ok 49 listing out files in 2154ms
ok 50 copying files around in 6525ms
ok 51 upconfig in 2592ms

> Task :solr:packaging:integrationTests FAILED

ERROR: The following test(s) have failed:
  - org.apache.solr.cloud.TestConfigSetsAPIExclusivity.testAPIExclusivity (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.TestConfigSetsAPIExclusivity.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.TestConfigSetsAPIExclusivity.testAPIExclusivity" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=D01445D7CE7006F0 -Ptests.file.encoding=US-ASCII

  - org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestCoordinatorRole.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=D01445D7CE7006F0 -Ptests.file.encoding=US-ASCII

  - org.apache.solr.search.TestCoordinatorRole.classMethod (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestCoordinatorRole.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.TestCoordinatorRole.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=D01445D7CE7006F0 -Ptests.file.encoding=US-ASCII

  - org.apache.solr.search.TestCoordinatorRole.classMethod (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestCoordinatorRole.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.TestCoordinatorRole.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=D01445D7CE7006F0 -Ptests.file.encoding=US-ASCII

  - org.apache.solr.cloud.RecoveryZkTest.test (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.RecoveryZkTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.RecoveryZkTest.test" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=D01445D7CE7006F0 -Ptests.file.encoding=US-ASCII

  - org.apache.solr.cloud.RecoveryZkTest.classMethod (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.RecoveryZkTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.RecoveryZkTest.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=D01445D7CE7006F0 -Ptests.file.encoding=US-ASCII


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:core:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/core/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 16s
625 actionable tasks: 609 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m41s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4947 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4947/

1 tests failed.
FAILED:  org.apache.solr.security.hadoop.LocaleTest.testWithKdc

Error Message:
org.apache.kerby.kerberos.kerb.KrbException: Failed to start DefaultKrbServer

Stack Trace:
org.apache.kerby.kerberos.kerb.KrbException: Failed to start DefaultKrbServer
	at __randomizedtesting.SeedInfo.seed([B7B650E5CB85A632:EC2FD867A23736D4]:0)
	at app//org.apache.kerby.kerberos.kerb.server.impl.AbstractInternalKdcServer.start(AbstractInternalKdcServer.java:84)
	at app//org.apache.kerby.kerberos.kerb.server.KdcServer.start(KdcServer.java:268)
	at app//org.apache.kerby.kerberos.kerb.server.SimpleKdcServer.start(SimpleKdcServer.java:163)
	at app//org.apache.hadoop.minikdc.MiniKdc.start(MiniKdc.java:284)
	at app//org.apache.solr.security.hadoop.LocaleTest.testWithKdc(LocaleTest.java:56)
	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
Caused by: java.net.BindException: Address already in use (Bind failed)
	at java.base/java.net.PlainSocketImpl.socketBind(Native Method)
	at java.base/java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:452)
	at java.base/java.net.ServerSocket.bind(ServerSocket.java:395)
	at java.base/java.net.ServerSocket.bind(ServerSocket.java:349)
	at org.apache.kerby.kerberos.kerb.transport.KdcNetwork.listen(KdcNetwork.java:64)
	at org.apache.kerby.kerberos.kerb.server.impl.DefaultInternalKdcServerImpl.doStart(DefaultInternalKdcServerImpl.java:67)
	at org.apache.kerby.kerberos.kerb.server.impl.AbstractInternalKdcServer.start(AbstractInternalKdcServer.java:82)
	... 49 more




Build Log:
[...truncated 1275 lines...]
ERROR: The following test(s) have failed:
  - org.apache.solr.security.hadoop.LocaleTest.testWithKdc (:solr:modules:hadoop-auth)
    Test output: /tmp/src/solr/solr/modules/hadoop-auth/build/test-results/test/outputs/OUTPUT-org.apache.solr.security.hadoop.LocaleTest.txt
    Reproduce with: gradlew :solr:modules:hadoop-auth:test --tests "org.apache.solr.security.hadoop.LocaleTest.testWithKdc" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=B7B650E5CB85A632 -Ptests.file.encoding=UTF-8


FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':solr:modules:hadoop-auth:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/modules/hadoop-auth/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 4m 33s
615 actionable tasks: 599 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 4m56s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4946 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4946/

All tests passed

Build Log:
[...truncated 1254 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22853ms
ok 43 post solr log into solr in 9080ms
ok 44 start solr with ssl in 20186ms
ok 45 start solr with ssl and auth in 22802ms
ok 46 SOLR-11740 check 'solr stop' connection in 28053ms
ok 47 status detects locally running solr in 14698ms
ok 48 status does not expose cli parameters to end user in 210ms
ok 49 listing out files in 2207ms
ok 50 copying files around in 6604ms
ok 51 upconfig in 2680ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 14m 4s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 14m27.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4945 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4945/

All tests passed

Build Log:
[...truncated 1251 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22435ms
ok 43 post solr log into solr in 9050ms
ok 44 start solr with ssl in 20403ms
ok 45 start solr with ssl and auth in 22411ms
ok 46 SOLR-11740 check 'solr stop' connection in 28729ms
ok 47 status detects locally running solr in 14404ms
ok 48 status does not expose cli parameters to end user in 214ms
ok 49 listing out files in 2186ms
ok 50 copying files around in 6682ms
ok 51 upconfig in 2675ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 5s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m27s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4944 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4944/

All tests passed

Build Log:
[...truncated 1251 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22588ms
ok 43 post solr log into solr in 9410ms
ok 44 start solr with ssl in 19796ms
ok 45 start solr with ssl and auth in 22398ms
ok 46 SOLR-11740 check 'solr stop' connection in 28197ms
ok 47 status detects locally running solr in 14738ms
ok 48 status does not expose cli parameters to end user in 216ms
ok 49 listing out files in 2206ms
ok 50 copying files around in 6668ms
ok 51 upconfig in 2681ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 5s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m28.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4943 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4943/

All tests passed

Build Log:
[...truncated 1252 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22633ms
ok 43 post solr log into solr in 8904ms
ok 44 start solr with ssl in 20042ms
ok 45 start solr with ssl and auth in 22112ms
ok 46 SOLR-11740 check 'solr stop' connection in 28404ms
ok 47 status detects locally running solr in 14395ms
ok 48 status does not expose cli parameters to end user in 213ms
ok 49 listing out files in 2163ms
ok 50 copying files around in 6511ms
ok 51 upconfig in 2638ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 15m 55s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m17s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4942 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4942/

1 tests failed.
FAILED:  org.apache.solr.cloud.MigrateReplicasTest.testGoodSpreadDuringAssignWithNoTarget

Error Message:
java.lang.AssertionError: The non-source node '127.0.0.1:34379_solr' has the wrong number of replicas after the migration expected:<2> but was:<1>

Stack Trace:
java.lang.AssertionError: The non-source node '127.0.0.1:34379_solr' has the wrong number of replicas after the migration expected:<2> but was:<1>
	at __randomizedtesting.SeedInfo.seed([69062C3D5A22C1E7:476DF8076F411093]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:647)
	at org.apache.solr.cloud.MigrateReplicasTest.testGoodSpreadDuringAssignWithNoTarget(MigrateReplicasTest.java:303)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)




Build Log:
[...truncated 3903 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22267ms
ok 43 post solr log into solr in 9319ms
ok 44 start solr with ssl in 19886ms
ok 45 start solr with ssl and auth in 22712ms
ok 46 SOLR-11740 check 'solr stop' connection in 28069ms
ok 47 status detects locally running solr in 14478ms
ok 48 status does not expose cli parameters to end user in 206ms
ok 49 listing out files in 2169ms
ok 50 copying files around in 6641ms
ok 51 upconfig in 2645ms

> Task :solr:packaging:integrationTests FAILED

ERROR: The following test(s) have failed:
  - org.apache.solr.cloud.MigrateReplicasTest.testGoodSpreadDuringAssignWithNoTarget (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.MigrateReplicasTest.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.MigrateReplicasTest.testGoodSpreadDuringAssignWithNoTarget" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=69062C3D5A22C1E7 -Ptests.file.encoding=US-ASCII


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:core:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/core/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 1s
625 actionable tasks: 609 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m22.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4941 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4941/

All tests passed

Build Log:
[...truncated 1263 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22383ms
ok 43 post solr log into solr in 9821ms
ok 44 start solr with ssl in 20660ms
ok 45 start solr with ssl and auth in 23585ms
ok 46 SOLR-11740 check 'solr stop' connection in 28486ms
ok 47 status detects locally running solr in 14792ms
ok 48 status does not expose cli parameters to end user in 218ms
ok 49 listing out files in 2183ms
ok 50 copying files around in 6760ms
ok 51 upconfig in 2661ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 13m 24s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 13m48s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4940 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4940/

All tests passed

Build Log:
[...truncated 1256 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22618ms
ok 43 post solr log into solr in 9408ms
ok 44 start solr with ssl in 20270ms
ok 45 start solr with ssl and auth in 22057ms
ok 46 SOLR-11740 check 'solr stop' connection in 28408ms
ok 47 status detects locally running solr in 14376ms
ok 48 status does not expose cli parameters to end user in 211ms
ok 49 listing out files in 2213ms
ok 50 copying files around in 6606ms
ok 51 upconfig in 2661ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 3s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m25.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4939 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4939/

3 tests failed.
FAILED:  org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess

Error Message:
java.lang.AssertionError: expected:<4> but was:<5>

Stack Trace:
java.lang.AssertionError: expected:<4> but was:<5>
	at __randomizedtesting.SeedInfo.seed([18E51CD1CC575CBB:D7F823D0ADD2A1E8]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:633)
	at org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess(TestCoordinatorRole.java:573)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)


FAILED:  org.apache.solr.search.TestCoordinatorRole.classMethod

Error Message:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from SUITE scope at org.apache.solr.search.TestCoordinatorRole: 
   1) Thread[id=421, name=TestCoordinatorRole-348-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=429, name=TestCoordinatorRole-348-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=431, name=TestCoordinatorRole-348-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=424, name=TestCoordinatorRole-348-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=430, name=TestCoordinatorRole-348-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=428, name=TestCoordinatorRole-348-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=419, name=TestCoordinatorRole-348-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=427, name=TestCoordinatorRole-348-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=420, name=TestCoordinatorRole-348-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=418, name=TestCoordinatorRole-348-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from SUITE scope at org.apache.solr.search.TestCoordinatorRole: 
   1) Thread[id=421, name=TestCoordinatorRole-348-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=429, name=TestCoordinatorRole-348-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=431, name=TestCoordinatorRole-348-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=424, name=TestCoordinatorRole-348-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=430, name=TestCoordinatorRole-348-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=428, name=TestCoordinatorRole-348-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=419, name=TestCoordinatorRole-348-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=427, name=TestCoordinatorRole-348-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=420, name=TestCoordinatorRole-348-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=418, name=TestCoordinatorRole-348-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
	at __randomizedtesting.SeedInfo.seed([18E51CD1CC575CBB]:0)


FAILED:  org.apache.solr.search.TestCoordinatorRole.classMethod

Error Message:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=424, name=TestCoordinatorRole-348-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=430, name=TestCoordinatorRole-348-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=421, name=TestCoordinatorRole-348-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=428, name=TestCoordinatorRole-348-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=419, name=TestCoordinatorRole-348-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=427, name=TestCoordinatorRole-348-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=420, name=TestCoordinatorRole-348-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=429, name=TestCoordinatorRole-348-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=431, name=TestCoordinatorRole-348-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=418, name=TestCoordinatorRole-348-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=424, name=TestCoordinatorRole-348-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=430, name=TestCoordinatorRole-348-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=421, name=TestCoordinatorRole-348-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=428, name=TestCoordinatorRole-348-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=419, name=TestCoordinatorRole-348-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=427, name=TestCoordinatorRole-348-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=420, name=TestCoordinatorRole-348-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=429, name=TestCoordinatorRole-348-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=431, name=TestCoordinatorRole-348-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=418, name=TestCoordinatorRole-348-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
	at __randomizedtesting.SeedInfo.seed([18E51CD1CC575CBB]:0)




Build Log:
[...truncated 9502 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 23024ms
ok 43 post solr log into solr in 9446ms
ok 44 start solr with ssl in 19792ms
ok 45 start solr with ssl and auth in 22345ms
ok 46 SOLR-11740 check 'solr stop' connection in 28412ms
ok 47 status detects locally running solr in 14666ms
ok 48 status does not expose cli parameters to end user in 214ms
ok 49 listing out files in 2196ms
ok 50 copying files around in 6615ms
ok 51 upconfig in 2669ms

> Task :solr:packaging:integrationTests FAILED

ERROR: The following test(s) have failed:
  - org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestCoordinatorRole.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=18E51CD1CC575CBB -Ptests.file.encoding=US-ASCII

  - org.apache.solr.search.TestCoordinatorRole.classMethod (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestCoordinatorRole.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.TestCoordinatorRole.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=18E51CD1CC575CBB -Ptests.file.encoding=US-ASCII

  - org.apache.solr.search.TestCoordinatorRole.classMethod (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestCoordinatorRole.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.TestCoordinatorRole.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=18E51CD1CC575CBB -Ptests.file.encoding=US-ASCII


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:core:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/core/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 7s
625 actionable tasks: 609 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m30s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4938 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4938/

All tests passed

Build Log:
[...truncated 1256 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 23609ms
ok 43 post solr log into solr in 9839ms
ok 44 start solr with ssl in 20314ms
ok 45 start solr with ssl and auth in 22994ms
ok 46 SOLR-11740 check 'solr stop' connection in 28607ms
ok 47 status detects locally running solr in 14469ms
ok 48 status does not expose cli parameters to end user in 226ms
ok 49 listing out files in 2243ms
ok 50 copying files around in 6887ms
ok 51 upconfig in 2806ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 43s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 17m10s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4937 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4937/

2 tests failed.
FAILED:  org.apache.solr.ltr.response.transform.TestInterleavingTransformer.interleavingTransformerWithOriginalRanking_shouldReturnInterleavingPickInTheResults

Error Message:
java.lang.Exception: Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([EED819C2C00EF6F9]:0)


FAILED:  org.apache.solr.ltr.response.transform.TestInterleavingTransformer.classMethod

Error Message:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([EED819C2C00EF6F9]:0)




Build Log:
[...truncated 1240 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22319ms
ok 43 post solr log into solr in 9125ms
ok 44 start solr with ssl in 19990ms
ok 45 start solr with ssl and auth in 22832ms
ok 46 SOLR-11740 check 'solr stop' connection in 28434ms
ok 47 status detects locally running solr in 14420ms
ok 48 status does not expose cli parameters to end user in 212ms
ok 49 listing out files in 2201ms
ok 50 copying files around in 6633ms
ok 51 upconfig in 2634ms

> Task :solr:packaging:integrationTests FAILED

> Task :solr:modules:ltr:test

org.apache.solr.ltr.response.transform.TestInterleavingTransformer > interleavingTransformerWithOriginalRanking_shouldReturnInterleavingPickInTheResults FAILED
    java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([EED819C2C00EF6F9]:0)

org.apache.solr.ltr.response.transform.TestInterleavingTransformer > classMethod FAILED
    java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([EED819C2C00EF6F9]:0)

org.apache.solr.ltr.response.transform.TestInterleavingTransformer > test suite's output saved to /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.response.transform.TestInterleavingTransformer.txt, copied below:
  1> 11:01:14.264 [TEST-TestInterleavingTransformer.interleavingTransformerWithOriginalRanking_shouldReturnInterleavingPickInTheResults-seed#[EED819C2C00EF6F9]] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
  2> июлы 21, 2023 2:01:12 ӔМБИСБОНЫ ФӔСТӔ com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
  2> WARNING: Suite execution timed out: org.apache.solr.ltr.response.transform.TestInterleavingTransformer
  2> ==== jstack at approximately timeout time ====
  2> "Test worker" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$2@380cfdf9
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$2@380cfdf9
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1300)
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1375)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:647)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:504)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:108)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:57)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:39)
  2> 	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
  2> 	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:52)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
  2> 	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
  2> 	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
  2> 	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
  2> 	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
  2> 
  2> "Reference Handler" ID=2 RUNNABLE
  2> 	at java.base@11.0.16/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
  2> 	at java.base@11.0.16/java.lang.ref.Reference.processPendingReferences(Reference.java:241)
  2> 	at java.base@11.0.16/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)
  2> 
  2> "Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@532e3361
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.ref.ReferenceQueue$Lock@532e3361
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
  2> 	at java.base@11.0.16/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)
  2> 
  2> "Signal Dispatcher" ID=4 RUNNABLE
  2> 
  2> "Common-Cleaner" ID=9 TIMED_WAITING on java.lang.ref.ReferenceQueue$Lock@7130b224
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- timed waiting on java.lang.ref.ReferenceQueue$Lock@7130b224
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	at java.base@11.0.16/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:161)
  2> 
  2> "/127.0.0.1:45054 to /127.0.0.1:39261 workers" ID=11 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7f1ff771
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@7f1ff771
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:403)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@42e99e4a
  2> 
  2> "/127.0.0.1:45054 to /127.0.0.1:39261 workers Thread 2" ID=12 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1fc044f2
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1fc044f2
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionDispatch.run(MessageHub.java:322)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@42a48628
  2> 
  2> "/127.0.0.1:45054 to /127.0.0.1:39261 workers Thread 3" ID=13 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@3d57d950
  2> 	- locked sun.nio.ch.EPollSelectorImpl@4902fd4c
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at org.gradle.internal.remote.internal.inet.SocketConnection$SocketInputStream.read(SocketConnection.java:185)
  2> 	at com.esotericsoftware.kryo.io.Input.fill(Input.java:146)
  2> 	at com.esotericsoftware.kryo.io.Input.require(Input.java:178)
  2> 	at com.esotericsoftware.kryo.io.Input.readByte(Input.java:295)
  2> 	at org.gradle.internal.serialize.kryo.KryoBackedDecoder.readByte(KryoBackedDecoder.java:88)
  2> 	at org.gradle.internal.remote.internal.hub.InterHubMessageSerializer$MessageReader.read(InterHubMessageSerializer.java:64)
  2> 	at org.gradle.internal.remote.internal.hub.InterHubMessageSerializer$MessageReader.read(InterHubMessageSerializer.java:52)
  2> 	at org.gradle.internal.remote.internal.inet.SocketConnection.receive(SocketConnection.java:81)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionReceive.run(MessageHub.java:270)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@74f0ea28
  2> 
  2> "SUITE-TestInterleavingTransformer-seed#[EED819C2C00EF6F9]" ID=14 RUNNABLE
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpThreads0(Native Method)
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:521)
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:509)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:708)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:59)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:437)
  2> 	- locked java.lang.Object@49a62f62
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:716)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:637)
  2> 
  2> "TEST-TestInterleavingTransformer.interleavingTransformerWithOriginalRanking_shouldReturnInterleavingPickInTheResults-seed#[EED819C2C00EF6F9]" ID=15 WAITING on java.util.concurrent.CountDownLatch$Sync@4ce75ec
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.CountDownLatch$Sync@4ce75ec
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
  2> 	at java.base@11.0.16/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
  2> 	at app//org.apache.solr.servlet.CoreContainerProvider$ContextInitializationKey.waitForReadyService(CoreContainerProvider.java:525)
  2> 	at app//org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:564)
  2> 	at app//org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
  2> 	at app//org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992)
  2> 	at app//org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427)
  2> 	- locked org.apache.solr.embedded.JettySolrRunner$1@54dc1570
  2> 	at app//org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253)
  2> 	at app//org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554)
  2> 	- locked org.apache.solr.embedded.JettySolrRunner@5742cbe2
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525)
  2> 	at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126)
  2> 	at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86)
  2> 	at app//org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57)
  2> 	at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181)
  2> 	at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:113)
  2> 	at app//org.apache.solr.ltr.response.transform.TestInterleavingTransformer.before(TestInterleavingTransformer.java:32)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:980)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2> 	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$$Lambda$177/0x00000008401edc40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@9f4a4db
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@715157b8
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@4512ab77
  2> 
  2> "qtp1547061655-17" ID=17 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@2830dbd2
  2> 	- locked sun.nio.ch.EPollSelectorImpl@53a7a5fb
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x000000084030f440.run(Unknown Source)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1547061655-18-acceptor-0@4cfb7001-ServerConnector@6dfaaa1e{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:34805}" ID=18 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
  2> 	at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
  2> 	at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@3bef5eb
  2> 
  2> "qtp1547061655-19" ID=19 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74a88165
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74a88165
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1547061655-20" ID=20 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74a88165
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74a88165
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1547061655-21" ID=21 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74a88165
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74a88165
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1547061655-22" ID=22 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74a88165
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74a88165
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1547061655-23" ID=23 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74a88165
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74a88165
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp1547061655-24" ID=24 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74a88165
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@74a88165
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "Session-HouseKeeper-2c3bb3cc-1" ID=25 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5ae82a69
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@5ae82a69
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
  2> 	at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "httpShardExecutor-6-thread-1" ID=26 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@340ec4b2
  2> 	- locked sun.nio.ch.EPollSelectorImpl@40eb6e75
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x000000084030f440.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x0000000840416440.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@13387294
  2> 
  2> "httpShardExecutor-6-thread-2" ID=27 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@70786ca4
  2> 	- locked sun.nio.ch.EPollSelectorImpl@6b58bba0
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x000000084030f440.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x0000000840416440.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@4ad8bc17
  2> 
  2> "Connection evictor" ID=28 TIMED_WAITING
  2> 	at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
  2> 	at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "Connection evictor" ID=29 TIMED_WAITING
  2> 	at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
  2> 	at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "h2sc-8-thread-1" ID=30 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@29b1d92d
  2> 	- locked sun.nio.ch.EPollSelectorImpl@323282a8
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x000000084030f440.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x0000000840416440.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@20e73208
  2> 
  2> "h2sc-8-thread-2" ID=31 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@19cd506
  2> 	- locked sun.nio.ch.EPollSelectorImpl@2f37fb2e
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x000000084030f440.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x0000000840416440.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@a1328b
  2> 
  2> "CloserThread" ID=33 WAITING on java.lang.Object@8e82754
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.Object@8e82754
  2> 	at java.base@11.0.16/java.lang.Object.wait(Object.java:328)
  2> 	at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2564)
  2> 
  2> "ForkJoinPool.commonPool-worker-3" ID=36 WAITING on java.util.concurrent.ForkJoinPool@2e69c323
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.ForkJoinPool@2e69c323
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
  2> 	at java.base@11.0.16/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
  2> 
  2> "searcherExecutor-13-thread-1" ID=37 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@493a070d
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@493a070d
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> ^^==============================================
  2> 
   >     java.lang.Exception: Test abandoned because suite timeout was reached.
   >         at __randomizedtesting.SeedInfo.seed([EED819C2C00EF6F9]:0)
  2> NOTE: reproduce with: gradlew test --tests TestInterleavingTransformer.interleavingTransformerWithOriginalRanking_shouldReturnInterleavingPickInTheResults -Dtests.seed=EED819C2C00EF6F9 -Dtests.locale=os -Dtests.timezone=Atlantic/Faeroe -Dtests.asserts=true -Dtests.file.encoding=UTF-8
  2> июлы 21, 2023 2:01:12 ӔМБИСБОНЫ ФӔСТӔ com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> WARNING: Will linger awaiting termination of 16 leaked thread(s).
  2> июлы 21, 2023 2:01:13 ӔМБИСБОНЫ ФӔСТӔ com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> SEVERE: 16 threads leaked from SUITE scope at org.apache.solr.ltr.response.transform.TestInterleavingTransformer: 
  2>    1) Thread[id=15, name=TEST-TestInterleavingTransformer.interleavingTransformerWithOriginalRanking_shouldReturnInterleavingPickInTheResults-seed#[EED819C2C00EF6F9], state=WAITING, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
  2>         at java.base@11.0.16/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
  2>         at app//org.apache.solr.servlet.CoreContainerProvider$ContextInitializationKey.waitForReadyService(CoreContainerProvider.java:525)
  2>         at app//org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:564)
  2>         at app//org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
  2>         at app//org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992)
  2>         at app//org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480)
  2>         at app//org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427)
  2>         at app//org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253)
  2>         at app//org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525)
  2>         at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126)
  2>         at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86)
  2>         at app//org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57)
  2>         at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181)
  2>         at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:113)
  2>         at app//org.apache.solr.ltr.response.transform.TestInterleavingTransformer.before(TestInterleavingTransformer.java:32)
  2>         at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2>         at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2>         at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2>         at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:980)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  2>         at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2>         at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  2>         at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2>         at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$$Lambda$177/0x00000008401edc40.run(Unknown Source)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    2) Thread[id=37, name=searcherExecutor-13-thread-1, state=WAITING, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2>         at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    3) Thread[id=19, name=qtp1547061655-19, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    4) Thread[id=31, name=h2sc-8-thread-2, state=RUNNABLE, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x000000084030f440.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x0000000840416440.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    5) Thread[id=26, name=httpShardExecutor-6-thread-1, state=RUNNABLE, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x000000084030f440.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x0000000840416440.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    6) Thread[id=20, name=qtp1547061655-20, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    7) Thread[id=22, name=qtp1547061655-22, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    8) Thread[id=30, name=h2sc-8-thread-1, state=RUNNABLE, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x000000084030f440.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x0000000840416440.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    9) Thread[id=18, name=qtp1547061655-18-acceptor-0@4cfb7001-ServerConnector@6dfaaa1e{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:34805}, state=RUNNABLE, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
  2>         at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
  2>         at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   10) Thread[id=24, name=qtp1547061655-24, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   11) Thread[id=33, name=CloserThread, state=WAITING, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2>         at java.base@11.0.16/java.lang.Object.wait(Object.java:328)
  2>         at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2564)
  2>   12) Thread[id=25, name=Session-HouseKeeper-2c3bb3cc-1, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
  2>         at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   13) Thread[id=17, name=qtp1547061655-17, state=RUNNABLE, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x000000084030f440.run(Unknown Source)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   14) Thread[id=27, name=httpShardExecutor-6-thread-2, state=RUNNABLE, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x000000084030f440.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x0000000840416440.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   15) Thread[id=21, name=qtp1547061655-21, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   16) Thread[id=23, name=qtp1547061655-23, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> июлы 21, 2023 2:01:13 ӔМБИСБОНЫ ФӔСТӔ com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFO: Starting to interrupt leaked threads:
  2>    1) Thread[id=15, name=TEST-TestInterleavingTransformer.interleavingTransformerWithOriginalRanking_shouldReturnInterleavingPickInTheResults-seed#[EED819C2C00EF6F9], state=WAITING, group=TGRP-TestInterleavingTransformer]
  2>    2) Thread[id=37, name=searcherExecutor-13-thread-1, state=WAITING, group=TGRP-TestInterleavingTransformer]
  2>    3) Thread[id=19, name=qtp1547061655-19, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>    4) Thread[id=31, name=h2sc-8-thread-2, state=RUNNABLE, group=TGRP-TestInterleavingTransformer]
  2>    5) Thread[id=26, name=httpShardExecutor-6-thread-1, state=RUNNABLE, group=TGRP-TestInterleavingTransformer]
  2>    6) Thread[id=20, name=qtp1547061655-20, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>    7) Thread[id=22, name=qtp1547061655-22, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>    8) Thread[id=30, name=h2sc-8-thread-1, state=RUNNABLE, group=TGRP-TestInterleavingTransformer]
  2>    9) Thread[id=18, name=qtp1547061655-18-acceptor-0@4cfb7001-ServerConnector@6dfaaa1e{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:34805}, state=RUNNABLE, group=TGRP-TestInterleavingTransformer]
  2>   10) Thread[id=24, name=qtp1547061655-24, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>   11) Thread[id=33, name=CloserThread, state=WAITING, group=TGRP-TestInterleavingTransformer]
  2>   12) Thread[id=25, name=Session-HouseKeeper-2c3bb3cc-1, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>   13) Thread[id=17, name=qtp1547061655-17, state=RUNNABLE, group=TGRP-TestInterleavingTransformer]
  2>   14) Thread[id=27, name=httpShardExecutor-6-thread-2, state=RUNNABLE, group=TGRP-TestInterleavingTransformer]
  2>   15) Thread[id=21, name=qtp1547061655-21, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  2>   16) Thread[id=23, name=qtp1547061655-23, state=TIMED_WAITING, group=TGRP-TestInterleavingTransformer]
  1> 13:01:13.732 [TEST-TestInterleavingTransformer.interleavingTransformerWithOriginalRanking_shouldReturnInterleavingPickInTheResults-seed#[EED819C2C00EF6F9]] ERROR org.apache.solr.servlet.SolrDispatchFilter - Could not start Dispatch Filter.
  1> org.apache.solr.common.SolrException: Interrupted while obtaining reference to CoreService
  1> 	at org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:571) ~[solr-core-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 84c2c3e1bd986bff87de56a6667323a85ce1e2d5 [snapshot build, details omitted]]
  1> 	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147) [solr-core-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 84c2c3e1bd986bff87de56a6667323a85ce1e2d5 [snapshot build, details omitted]]
  1> 	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 84c2c3e1bd986bff87de56a6667323a85ce1e2d5 [snapshot build, details omitted]]
  1> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253) [jetty-util-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94) [jetty-util-10.0.15.jar:10.0.15]
  1> 	at org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 84c2c3e1bd986bff87de56a6667323a85ce1e2d5 [snapshot build, details omitted]]
  1> 	at org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 84c2c3e1bd986bff87de56a6667323a85ce1e2d5 [snapshot build, details omitted]]
  1> 	at org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 84c2c3e1bd986bff87de56a6667323a85ce1e2d5 [snapshot build, details omitted]]
  1> 	at org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 84c2c3e1bd986bff87de56a6667323a85ce1e2d5 [snapshot build, details omitted]]
  1> 	at org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 84c2c3e1bd986bff87de56a6667323a85ce1e2d5 [snapshot build, details omitted]]
  1> 	at org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT 84c2c3e1bd986bff87de56a6667323a85ce1e2d5 [snapshot build, details omitted]]
  1> 	at org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181) [test/:?]
  1> 	at org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:113) [test/:?]
  1> 	at org.apache.solr.ltr.response.transform.TestInterleavingTransformer.before(TestInterleavingTransformer.java:32) [test/:?]
  1> 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
  1> 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
  1> 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
  1> 	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:980) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at java.lang.Thread.run(Thread.java:829) [?:?]
  2> NOTE: leaving temporary files on disk at: /tmp/src/solr/solr/modules/ltr/build/tmp/tests-tmp/solr.ltr.response.transform.TestInterleavingTransformer_EED819C2C00EF6F9-001
  2> Jul 21, 2023 1:01:14 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFO: All leaked threads terminated.
   >     java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   >         at __randomizedtesting.SeedInfo.seed([EED819C2C00EF6F9]:0)
  2> NOTE: test params are: codec=Asserting(Lucene95): {}, docValues:{}, maxPointsInLeafNode=347, maxMBSortInHeap=6.224271728792102, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=os, timezone=Atlantic/Faeroe
  2> NOTE: Linux 5.4.0-100-generic amd64/Oracle Corporation 11.0.16 (64-bit)/cpus=1,threads=1,free=259010624,total=329252864
  2> NOTE: All tests run in this JVM: [TestInterleavingTransformer]
  2> NOTE: reproduce with: gradlew test --tests TestInterleavingTransformer -Dtests.seed=EED819C2C00EF6F9 -Dtests.locale=os -Dtests.timezone=Atlantic/Faeroe -Dtests.asserts=true -Dtests.file.encoding=UTF-8

:solr:modules:ltr:test (FAILURE): 184 test(s), 2 failure(s)

184 tests completed, 2 failed

> Task :solr:modules:ltr:test FAILED
> Task :solr:modules:ltr:wipeTaskTemp

ERROR: The following test(s) have failed:
  - org.apache.solr.ltr.response.transform.TestInterleavingTransformer.interleavingTransformerWithOriginalRanking_shouldReturnInterleavingPickInTheResults (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.response.transform.TestInterleavingTransformer.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.response.transform.TestInterleavingTransformer.interleavingTransformerWithOriginalRanking_shouldReturnInterleavingPickInTheResults" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=EED819C2C00EF6F9 -Ptests.file.encoding=US-ASCII

  - org.apache.solr.ltr.response.transform.TestInterleavingTransformer.classMethod (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.response.transform.TestInterleavingTransformer.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.response.transform.TestInterleavingTransformer.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=EED819C2C00EF6F9 -Ptests.file.encoding=US-ASCII


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:modules:ltr:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/modules/ltr/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 2h 2m 44s
642 actionable tasks: 626 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 2h3m9s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4936 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4936/

All tests passed

Build Log:
[...truncated 1254 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22779ms
ok 43 post solr log into solr in 9257ms
ok 44 start solr with ssl in 19969ms
ok 45 start solr with ssl and auth in 23063ms
ok 46 SOLR-11740 check 'solr stop' connection in 28493ms
ok 47 status detects locally running solr in 14655ms
ok 48 status does not expose cli parameters to end user in 211ms
ok 49 listing out files in 2169ms
ok 50 copying files around in 6566ms
ok 51 upconfig in 2639ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 8s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m31s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4935 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4935/

All tests passed

Build Log:
[...truncated 1254 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22598ms
ok 43 post solr log into solr in 9020ms
ok 44 start solr with ssl in 19813ms
ok 45 start solr with ssl and auth in 22446ms
ok 46 SOLR-11740 check 'solr stop' connection in 28120ms
ok 47 status detects locally running solr in 14731ms
ok 48 status does not expose cli parameters to end user in 218ms
ok 49 listing out files in 2198ms
ok 50 copying files around in 6579ms
ok 51 upconfig in 2630ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 15m 59s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m20.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4934 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4934/

All tests passed

Build Log:
[...truncated 1259 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22878ms
ok 43 post solr log into solr in 9408ms
ok 44 start solr with ssl in 20243ms
ok 45 start solr with ssl and auth in 22866ms
ok 46 SOLR-11740 check 'solr stop' connection in 28684ms
ok 47 status detects locally running solr in 14641ms
ok 48 status does not expose cli parameters to end user in 210ms
ok 49 listing out files in 2169ms
ok 50 copying files around in 6630ms
ok 51 upconfig in 2659ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 13m 16s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 13m39s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4933 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4933/

All tests passed

Build Log:
[...truncated 1255 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22555ms
ok 43 post solr log into solr in 8938ms
ok 44 start solr with ssl in 20012ms
ok 45 start solr with ssl and auth in 22479ms
ok 46 SOLR-11740 check 'solr stop' connection in 28109ms
ok 47 status detects locally running solr in 14378ms
ok 48 status does not expose cli parameters to end user in 213ms
ok 49 listing out files in 2176ms
ok 50 copying files around in 6556ms
ok 51 upconfig in 2639ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 4s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m31.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4932 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4932/

All tests passed

Build Log:
[...truncated 1247 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22843ms
ok 43 post solr log into solr in 9320ms
ok 44 start solr with ssl in 20032ms
ok 45 start solr with ssl and auth in 22817ms
ok 46 SOLR-11740 check 'solr stop' connection in 28445ms
ok 47 status detects locally running solr in 14429ms
ok 48 status does not expose cli parameters to end user in 216ms
ok 49 listing out files in 2194ms
ok 50 copying files around in 6607ms
ok 51 upconfig in 2660ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 7s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m29s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4931 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4931/

All tests passed

Build Log:
[...truncated 1251 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22827ms
ok 43 post solr log into solr in 9205ms
ok 44 start solr with ssl in 19490ms
ok 45 start solr with ssl and auth in 22736ms
ok 46 SOLR-11740 check 'solr stop' connection in 28346ms
ok 47 status detects locally running solr in 14349ms
ok 48 status does not expose cli parameters to end user in 208ms
ok 49 listing out files in 2162ms
ok 50 copying files around in 6531ms
ok 51 upconfig in 2636ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 15m 59s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m21s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4930 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4930/

No tests ran.

Build Log:
[...truncated 478 lines...]
BUILD FAILED in 1m 23s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m43.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4929 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4929/

No tests ran.

Build Log:
[...truncated 477 lines...]
BUILD FAILED in 1m 21s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m41s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4928 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4928/

No tests ran.

Build Log:
[...truncated 477 lines...]
BUILD FAILED in 1m 23s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m44s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4927 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4927/

No tests ran.

Build Log:
[...truncated 477 lines...]
BUILD FAILED in 1m 24s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m44s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4926 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4926/

No tests ran.

Build Log:
[...truncated 477 lines...]
BUILD FAILED in 1m 24s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m45s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4925 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4925/

No tests ran.

Build Log:
[...truncated 478 lines...]
BUILD FAILED in 1m 22s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m44s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4924 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4924/

No tests ran.

Build Log:
[...truncated 478 lines...]
BUILD FAILED in 1m 23s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m44s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4923 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4923/

No tests ran.

Build Log:
[...truncated 478 lines...]
BUILD FAILED in 1m 22s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m55s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4922 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4922/

No tests ran.

Build Log:
[...truncated 477 lines...]
BUILD FAILED in 1m 25s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m48.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4921 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4921/

No tests ran.

Build Log:
[...truncated 477 lines...]
BUILD FAILED in 1m 25s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m51.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4920 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4920/

No tests ran.

Build Log:
[...truncated 478 lines...]
BUILD FAILED in 1m 22s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m46.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4919 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4919/

No tests ran.

Build Log:
[...truncated 477 lines...]
BUILD FAILED in 1m 25s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m49s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4918 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4918/

No tests ran.

Build Log:
[...truncated 477 lines...]
BUILD FAILED in 1m 24s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m49.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4917 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4917/

No tests ran.

Build Log:
[...truncated 477 lines...]
BUILD FAILED in 1m 25s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m49s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4916 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4916/

No tests ran.

Build Log:
[...truncated 477 lines...]
BUILD FAILED in 1m 25s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m48.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4915 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4915/

No tests ran.

Build Log:
[...truncated 489 lines...]
BUILD FAILED in 1m 24s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m48s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4914 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4914/

No tests ran.

Build Log:
[...truncated 474 lines...]
BUILD FAILED in 1m 25s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m53s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4913 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4913/

No tests ran.

Build Log:
[...truncated 477 lines...]
BUILD FAILED in 1m 26s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m53s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4912 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4912/

No tests ran.

Build Log:
[...truncated 478 lines...]
BUILD FAILED in 1m 23s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m49s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4911 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4911/

No tests ran.

Build Log:
[...truncated 477 lines...]
BUILD FAILED in 1m 25s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m55.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4910 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4910/

No tests ran.

Build Log:
[...truncated 478 lines...]
BUILD FAILED in 1m 26s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 1m50.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4909 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4909/

No tests ran.

Build Log:
[...truncated 473 lines...]
BUILD FAILED in 1m 41s
212 actionable tasks: 196 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 2m12.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4908 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4908/

3 tests failed.
FAILED:  org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess

Error Message:
java.lang.AssertionError: expected:<4> but was:<5>

Stack Trace:
java.lang.AssertionError: expected:<4> but was:<5>
	at __randomizedtesting.SeedInfo.seed([A0860CA4F8B40FF:C5155FCB2E0EBDAC]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:633)
	at org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess(TestCoordinatorRole.java:573)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)


FAILED:  org.apache.solr.search.TestCoordinatorRole.classMethod

Error Message:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from SUITE scope at org.apache.solr.search.TestCoordinatorRole: 
   1) Thread[id=1129, name=TestCoordinatorRole-847-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=1127, name=TestCoordinatorRole-847-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=1132, name=TestCoordinatorRole-847-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=1134, name=TestCoordinatorRole-847-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=1126, name=TestCoordinatorRole-847-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=1130, name=TestCoordinatorRole-847-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=1131, name=TestCoordinatorRole-847-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=1135, name=TestCoordinatorRole-847-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=1133, name=TestCoordinatorRole-847-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=1128, name=TestCoordinatorRole-847-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from SUITE scope at org.apache.solr.search.TestCoordinatorRole: 
   1) Thread[id=1129, name=TestCoordinatorRole-847-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=1127, name=TestCoordinatorRole-847-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=1132, name=TestCoordinatorRole-847-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=1134, name=TestCoordinatorRole-847-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=1126, name=TestCoordinatorRole-847-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=1130, name=TestCoordinatorRole-847-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=1131, name=TestCoordinatorRole-847-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=1135, name=TestCoordinatorRole-847-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=1133, name=TestCoordinatorRole-847-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=1128, name=TestCoordinatorRole-847-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
	at __randomizedtesting.SeedInfo.seed([A0860CA4F8B40FF]:0)


FAILED:  org.apache.solr.search.TestCoordinatorRole.classMethod

Error Message:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=1129, name=TestCoordinatorRole-847-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=1132, name=TestCoordinatorRole-847-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=1134, name=TestCoordinatorRole-847-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=1127, name=TestCoordinatorRole-847-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=1126, name=TestCoordinatorRole-847-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=1130, name=TestCoordinatorRole-847-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=1131, name=TestCoordinatorRole-847-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=1135, name=TestCoordinatorRole-847-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=1133, name=TestCoordinatorRole-847-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=1128, name=TestCoordinatorRole-847-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=1129, name=TestCoordinatorRole-847-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=1132, name=TestCoordinatorRole-847-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=1134, name=TestCoordinatorRole-847-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=1127, name=TestCoordinatorRole-847-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=1126, name=TestCoordinatorRole-847-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=1130, name=TestCoordinatorRole-847-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=1131, name=TestCoordinatorRole-847-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=1135, name=TestCoordinatorRole-847-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=1133, name=TestCoordinatorRole-847-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=1128, name=TestCoordinatorRole-847-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
	at __randomizedtesting.SeedInfo.seed([A0860CA4F8B40FF]:0)




Build Log:
[...truncated 9756 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22257ms
ok 43 post solr log into solr in 9453ms
ok 44 start solr with ssl in 20468ms
ok 45 start solr with ssl and auth in 22176ms
ok 46 SOLR-11740 check 'solr stop' connection in 28122ms
ok 47 status detects locally running solr in 14654ms
ok 48 status does not expose cli parameters to end user in 209ms
ok 49 listing out files in 2161ms
ok 50 copying files around in 6511ms
ok 51 upconfig in 2635ms

> Task :solr:packaging:integrationTests FAILED

ERROR: The following test(s) have failed:
  - org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestCoordinatorRole.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=A0860CA4F8B40FF -Ptests.file.encoding=ISO-8859-1

  - org.apache.solr.search.TestCoordinatorRole.classMethod (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestCoordinatorRole.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.TestCoordinatorRole.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=A0860CA4F8B40FF -Ptests.file.encoding=ISO-8859-1

  - org.apache.solr.search.TestCoordinatorRole.classMethod (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestCoordinatorRole.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.TestCoordinatorRole.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=A0860CA4F8B40FF -Ptests.file.encoding=ISO-8859-1


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:core:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/core/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 15m 50s
625 actionable tasks: 609 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m18s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4907 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4907/

2 tests failed.
FAILED:  org.apache.solr.ltr.model.TestDefaultWrapperModel.testLoadNestedWrapperModel

Error Message:
java.lang.Exception: Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([713F09F711D0E46F]:0)


FAILED:  org.apache.solr.ltr.model.TestDefaultWrapperModel.classMethod

Error Message:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([713F09F711D0E46F]:0)




Build Log:
[...truncated 1245 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 23089ms
ok 43 post solr log into solr in 9096ms
ok 44 start solr with ssl in 20274ms
ok 45 start solr with ssl and auth in 22232ms
ok 46 SOLR-11740 check 'solr stop' connection in 28181ms
ok 47 status detects locally running solr in 14442ms
ok 48 status does not expose cli parameters to end user in 217ms
ok 49 listing out files in 2192ms
ok 50 copying files around in 6710ms
ok 51 upconfig in 2697ms

> Task :solr:packaging:integrationTests FAILED

> Task :solr:modules:ltr:test

org.apache.solr.ltr.model.TestDefaultWrapperModel > testLoadNestedWrapperModel FAILED
    java.lang.Exception: Test abandoned because suite timeout was reached.
        at __randomizedtesting.SeedInfo.seed([713F09F711D0E46F]:0)

org.apache.solr.ltr.model.TestDefaultWrapperModel > classMethod FAILED
    java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([713F09F711D0E46F]:0)

org.apache.solr.ltr.model.TestDefaultWrapperModel > test suite's output saved to /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.model.TestDefaultWrapperModel.txt, copied below:
  1> 09:40:30.895 [TEST-TestDefaultWrapperModel.testLoadNestedWrapperModel-seed#[713F09F711D0E46F]] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
  2> iməg àdùmbə̀ŋ 18, 2023 8:40:29 AM com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
  2> WARNING: Suite execution timed out: org.apache.solr.ltr.model.TestDefaultWrapperModel
  2> ==== jstack at approximately timeout time ====
  2> "Test worker" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$2@4ba1da1a
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$2@4ba1da1a
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1300)
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1375)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:647)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:504)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:108)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:57)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:39)
  2> 	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
  2> 	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:52)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
  2> 	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
  2> 	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
  2> 	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
  2> 	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
  2> 
  2> "Reference Handler" ID=2 RUNNABLE
  2> 	at java.base@11.0.16/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
  2> 	at java.base@11.0.16/java.lang.ref.Reference.processPendingReferences(Reference.java:241)
  2> 	at java.base@11.0.16/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)
  2> 
  2> "Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@1e1fd445
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.ref.ReferenceQueue$Lock@1e1fd445
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
  2> 	at java.base@11.0.16/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)
  2> 
  2> "Signal Dispatcher" ID=4 RUNNABLE
  2> 
  2> "Common-Cleaner" ID=9 TIMED_WAITING on java.lang.ref.ReferenceQueue$Lock@7cc9b279
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- timed waiting on java.lang.ref.ReferenceQueue$Lock@7cc9b279
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	at java.base@11.0.16/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:161)
  2> 
  2> "/127.0.0.1:40306 to /127.0.0.1:39761 workers" ID=11 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3637b323
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3637b323
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:403)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@42e99e4a
  2> 
  2> "/127.0.0.1:40306 to /127.0.0.1:39761 workers Thread 2" ID=12 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@775c2e70
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@775c2e70
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionDispatch.run(MessageHub.java:322)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@42a48628
  2> 
  2> "/127.0.0.1:40306 to /127.0.0.1:39761 workers Thread 3" ID=13 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@17016122
  2> 	- locked sun.nio.ch.EPollSelectorImpl@266a53c1
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at org.gradle.internal.remote.internal.inet.SocketConnection$SocketInputStream.read(SocketConnection.java:185)
  2> 	at com.esotericsoftware.kryo.io.Input.fill(Input.java:146)
  2> 	at com.esotericsoftware.kryo.io.Input.require(Input.java:178)
  2> 	at com.esotericsoftware.kryo.io.Input.readByte(Input.java:295)
  2> 	at org.gradle.internal.serialize.kryo.KryoBackedDecoder.readByte(KryoBackedDecoder.java:88)
  2> 	at org.gradle.internal.remote.internal.hub.InterHubMessageSerializer$MessageReader.read(InterHubMessageSerializer.java:64)
  2> 	at org.gradle.internal.remote.internal.hub.InterHubMessageSerializer$MessageReader.read(InterHubMessageSerializer.java:52)
  2> 	at org.gradle.internal.remote.internal.inet.SocketConnection.receive(SocketConnection.java:81)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionReceive.run(MessageHub.java:270)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@74f0ea28
  2> 
  2> "SUITE-TestDefaultWrapperModel-seed#[713F09F711D0E46F]" ID=14 RUNNABLE
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpThreads0(Native Method)
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:521)
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:509)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:708)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:59)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:437)
  2> 	- locked java.lang.Object@2d4f4284
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:716)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:637)
  2> 
  2> "TEST-TestDefaultWrapperModel.testLoadNestedWrapperModel-seed#[713F09F711D0E46F]" ID=15 WAITING on java.util.concurrent.CountDownLatch$Sync@3bb4c2b0
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.CountDownLatch$Sync@3bb4c2b0
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
  2> 	at java.base@11.0.16/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
  2> 	at app//org.apache.solr.servlet.CoreContainerProvider$ContextInitializationKey.waitForReadyService(CoreContainerProvider.java:525)
  2> 	at app//org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:564)
  2> 	at app//org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
  2> 	at app//org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992)
  2> 	at app//org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427)
  2> 	- locked org.apache.solr.embedded.JettySolrRunner$1@3bf6c6f3
  2> 	at app//org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253)
  2> 	at app//org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554)
  2> 	- locked org.apache.solr.embedded.JettySolrRunner@2762a8de
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525)
  2> 	at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126)
  2> 	at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86)
  2> 	at app//org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57)
  2> 	at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181)
  2> 	at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:113)
  2> 	at app//org.apache.solr.ltr.model.TestDefaultWrapperModel.setupBeforeClass(TestDefaultWrapperModel.java:46)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:980)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2> 	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$$Lambda$177/0x00000008401edc40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@536203a8
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@46e2022
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@9a2af02
  2> 
  2> "qtp138335743-17" ID=17 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@7808e92d
  2> 	- locked sun.nio.ch.EPollSelectorImpl@d92b93e
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x00000008402fc440.run(Unknown Source)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp138335743-18-acceptor-0@9feb625-ServerConnector@54b1f7f1{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:40961}" ID=18 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
  2> 	at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
  2> 	at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@11d4cd31
  2> 
  2> "qtp138335743-19" ID=19 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@c682cf9
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@c682cf9
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp138335743-20" ID=20 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@c682cf9
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@c682cf9
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp138335743-21" ID=21 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@c682cf9
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@c682cf9
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp138335743-22" ID=22 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@c682cf9
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@c682cf9
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp138335743-23" ID=23 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@c682cf9
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@c682cf9
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp138335743-24" ID=24 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@c682cf9
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@c682cf9
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "Session-HouseKeeper-35af148d-1" ID=25 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3316edd8
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@3316edd8
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
  2> 	at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "httpShardExecutor-6-thread-1" ID=26 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@2b55f9cd
  2> 	- locked sun.nio.ch.EPollSelectorImpl@1c5f04d3
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x00000008402fc440.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@60eab70d
  2> 
  2> "httpShardExecutor-6-thread-2" ID=27 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@511dfc45
  2> 	- locked sun.nio.ch.EPollSelectorImpl@55e1b393
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x00000008402fc440.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@6daf423f
  2> 
  2> "Connection evictor" ID=28 TIMED_WAITING
  2> 	at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
  2> 	at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "Connection evictor" ID=29 TIMED_WAITING
  2> 	at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
  2> 	at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "h2sc-8-thread-1" ID=30 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@51feb373
  2> 	- locked sun.nio.ch.EPollSelectorImpl@35e34c03
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x00000008402fc440.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@632a8bb0
  2> 
  2> "h2sc-8-thread-2" ID=31 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@b22712e
  2> 	- locked sun.nio.ch.EPollSelectorImpl@530736f2
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x00000008402fc440.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@695eab7b
  2> 
  2> "CloserThread" ID=33 WAITING on java.lang.Object@774662ca
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.Object@774662ca
  2> 	at java.base@11.0.16/java.lang.Object.wait(Object.java:328)
  2> 	at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2564)
  2> 
  2> "ForkJoinPool.commonPool-worker-3" ID=36 WAITING on java.util.concurrent.ForkJoinPool@586aab75
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.ForkJoinPool@586aab75
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
  2> 	at java.base@11.0.16/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
  2> 
  2> "searcherExecutor-13-thread-1" ID=37 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@40590a57
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@40590a57
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> ^^==============================================
  2> 
   >     java.lang.Exception: Test abandoned because suite timeout was reached.
   >         at __randomizedtesting.SeedInfo.seed([713F09F711D0E46F]:0)
  2> NOTE: reproduce with: gradlew test --tests TestDefaultWrapperModel.testLoadNestedWrapperModel -Dtests.seed=713F09F711D0E46F -Dtests.locale=mgo -Dtests.timezone=America/Sao_Paulo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
  2> iməg àdùmbə̀ŋ 18, 2023 8:40:29 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> WARNING: Will linger awaiting termination of 16 leaked thread(s).
  2> iməg àdùmbə̀ŋ 18, 2023 8:40:30 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> SEVERE: 16 threads leaked from SUITE scope at org.apache.solr.ltr.model.TestDefaultWrapperModel: 
  2>    1) Thread[id=17, name=qtp138335743-17, state=RUNNABLE, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x00000008402fc440.run(Unknown Source)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    2) Thread[id=22, name=qtp138335743-22, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    3) Thread[id=25, name=Session-HouseKeeper-35af148d-1, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
  2>         at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    4) Thread[id=31, name=h2sc-8-thread-2, state=RUNNABLE, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x00000008402fc440.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    5) Thread[id=20, name=qtp138335743-20, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    6) Thread[id=23, name=qtp138335743-23, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    7) Thread[id=27, name=httpShardExecutor-6-thread-2, state=RUNNABLE, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x00000008402fc440.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    8) Thread[id=15, name=TEST-TestDefaultWrapperModel.testLoadNestedWrapperModel-seed#[713F09F711D0E46F], state=WAITING, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
  2>         at java.base@11.0.16/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
  2>         at app//org.apache.solr.servlet.CoreContainerProvider$ContextInitializationKey.waitForReadyService(CoreContainerProvider.java:525)
  2>         at app//org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:564)
  2>         at app//org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
  2>         at app//org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992)
  2>         at app//org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480)
  2>         at app//org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427)
  2>         at app//org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253)
  2>         at app//org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525)
  2>         at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126)
  2>         at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86)
  2>         at app//org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57)
  2>         at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181)
  2>         at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:113)
  2>         at app//org.apache.solr.ltr.model.TestDefaultWrapperModel.setupBeforeClass(TestDefaultWrapperModel.java:46)
  2>         at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2>         at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2>         at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2>         at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:980)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  2>         at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2>         at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  2>         at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2>         at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$$Lambda$177/0x00000008401edc40.run(Unknown Source)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    9) Thread[id=26, name=httpShardExecutor-6-thread-1, state=RUNNABLE, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x00000008402fc440.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   10) Thread[id=33, name=CloserThread, state=WAITING, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2>         at java.base@11.0.16/java.lang.Object.wait(Object.java:328)
  2>         at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2564)
  2>   11) Thread[id=24, name=qtp138335743-24, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   12) Thread[id=19, name=qtp138335743-19, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   13) Thread[id=21, name=qtp138335743-21, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   14) Thread[id=30, name=h2sc-8-thread-1, state=RUNNABLE, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$252/0x00000008402fc440.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$352/0x000000084040e440.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   15) Thread[id=37, name=searcherExecutor-13-thread-1, state=WAITING, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2>         at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   16) Thread[id=18, name=qtp138335743-18-acceptor-0@9feb625-ServerConnector@54b1f7f1{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:40961}, state=RUNNABLE, group=TGRP-TestDefaultWrapperModel]
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
  2>         at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
  2>         at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> iməg àdùmbə̀ŋ 18, 2023 8:40:30 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFO: Starting to interrupt leaked threads:
  2>    1) Thread[id=17, name=qtp138335743-17, state=RUNNABLE, group=TGRP-TestDefaultWrapperModel]
  2>    2) Thread[id=22, name=qtp138335743-22, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>    3) Thread[id=25, name=Session-HouseKeeper-35af148d-1, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>    4) Thread[id=31, name=h2sc-8-thread-2, state=RUNNABLE, group=TGRP-TestDefaultWrapperModel]
  2>    5) Thread[id=20, name=qtp138335743-20, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>    6) Thread[id=23, name=qtp138335743-23, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>    7) Thread[id=27, name=httpShardExecutor-6-thread-2, state=RUNNABLE, group=TGRP-TestDefaultWrapperModel]
  2>    8) Thread[id=15, name=TEST-TestDefaultWrapperModel.testLoadNestedWrapperModel-seed#[713F09F711D0E46F], state=WAITING, group=TGRP-TestDefaultWrapperModel]
  2>    9) Thread[id=26, name=httpShardExecutor-6-thread-1, state=RUNNABLE, group=TGRP-TestDefaultWrapperModel]
  2>   10) Thread[id=33, name=CloserThread, state=WAITING, group=TGRP-TestDefaultWrapperModel]
  2>   11) Thread[id=24, name=qtp138335743-24, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>   12) Thread[id=19, name=qtp138335743-19, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>   13) Thread[id=21, name=qtp138335743-21, state=TIMED_WAITING, group=TGRP-TestDefaultWrapperModel]
  2>   14) Thread[id=30, name=h2sc-8-thread-1, state=RUNNABLE, group=TGRP-TestDefaultWrapperModel]
  2>   15) Thread[id=37, name=searcherExecutor-13-thread-1, state=WAITING, group=TGRP-TestDefaultWrapperModel]
  2>   16) Thread[id=18, name=qtp138335743-18-acceptor-0@9feb625-ServerConnector@54b1f7f1{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:40961}, state=RUNNABLE, group=TGRP-TestDefaultWrapperModel]
  1> 11:40:30.364 [TEST-TestDefaultWrapperModel.testLoadNestedWrapperModel-seed#[713F09F711D0E46F]] ERROR org.apache.solr.servlet.SolrDispatchFilter - Could not start Dispatch Filter.
  1> org.apache.solr.common.SolrException: Interrupted while obtaining reference to CoreService
  1> 	at org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:571) ~[solr-core-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT a99e0bc96f0dfdaac235a0d4685dc232d4d7b341 [snapshot build, details omitted]]
  1> 	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147) [solr-core-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT a99e0bc96f0dfdaac235a0d4685dc232d4d7b341 [snapshot build, details omitted]]
  1> 	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT a99e0bc96f0dfdaac235a0d4685dc232d4d7b341 [snapshot build, details omitted]]
  1> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253) [jetty-util-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94) [jetty-util-10.0.15.jar:10.0.15]
  1> 	at org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT a99e0bc96f0dfdaac235a0d4685dc232d4d7b341 [snapshot build, details omitted]]
  1> 	at org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT a99e0bc96f0dfdaac235a0d4685dc232d4d7b341 [snapshot build, details omitted]]
  1> 	at org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT a99e0bc96f0dfdaac235a0d4685dc232d4d7b341 [snapshot build, details omitted]]
  1> 	at org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT a99e0bc96f0dfdaac235a0d4685dc232d4d7b341 [snapshot build, details omitted]]
  1> 	at org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT a99e0bc96f0dfdaac235a0d4685dc232d4d7b341 [snapshot build, details omitted]]
  1> 	at org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57) [solr-test-framework-9.4.0-SNAPSHOT.jar:9.4.0-SNAPSHOT a99e0bc96f0dfdaac235a0d4685dc232d4d7b341 [snapshot build, details omitted]]
  1> 	at org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181) [test/:?]
  1> 	at org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:113) [test/:?]
  1> 	at org.apache.solr.ltr.model.TestDefaultWrapperModel.setupBeforeClass(TestDefaultWrapperModel.java:46) [test/:?]
  1> 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
  1> 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
  1> 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
  1> 	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:980) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) [lucene-test-framework-9.7.0.jar:9.7.0 ccf4b198ec328095d45d2746189dc8ca633e8bcf - 2023-06-21 11:48:16]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at java.lang.Thread.run(Thread.java:829) [?:?]
  2> NOTE: leaving temporary files on disk at: /tmp/src/solr/solr/modules/ltr/build/tmp/tests-tmp/solr.ltr.model.TestDefaultWrapperModel_713F09F711D0E46F-001
  2> Jul 18, 2023 11:40:31 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFO: All leaked threads terminated.
   >     java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   >         at __randomizedtesting.SeedInfo.seed([713F09F711D0E46F]:0)
  2> NOTE: test params are: codec=Asserting(Lucene95), sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=mgo, timezone=America/Sao_Paulo
  2> NOTE: Linux 5.4.0-100-generic amd64/Oracle Corporation 11.0.16 (64-bit)/cpus=1,threads=1,free=254130368,total=324534272
  2> NOTE: All tests run in this JVM: [TestDefaultWrapperModel]
  2> NOTE: reproduce with: gradlew test --tests TestDefaultWrapperModel -Dtests.seed=713F09F711D0E46F -Dtests.locale=mgo -Dtests.timezone=America/Sao_Paulo -Dtests.asserts=true -Dtests.file.encoding=UTF-8

:solr:modules:ltr:test (FAILURE): 184 test(s), 2 failure(s)

184 tests completed, 2 failed

> Task :solr:modules:ltr:test FAILED
> Task :solr:modules:ltr:wipeTaskTemp

ERROR: The following test(s) have failed:
  - org.apache.solr.ltr.model.TestDefaultWrapperModel.testLoadNestedWrapperModel (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.model.TestDefaultWrapperModel.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.model.TestDefaultWrapperModel.testLoadNestedWrapperModel" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=713F09F711D0E46F -Ptests.file.encoding=UTF-8

  - org.apache.solr.ltr.model.TestDefaultWrapperModel.classMethod (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.model.TestDefaultWrapperModel.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.model.TestDefaultWrapperModel.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=713F09F711D0E46F -Ptests.file.encoding=UTF-8


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:modules:ltr:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/modules/ltr/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 2h 2m 34s
642 actionable tasks: 626 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 2h2m59.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4906 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4906/

All tests passed

Build Log:
[...truncated 1257 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22184ms
ok 43 post solr log into solr in 9058ms
ok 44 start solr with ssl in 19552ms
ok 45 start solr with ssl and auth in 21861ms
ok 46 SOLR-11740 check 'solr stop' connection in 28551ms
ok 47 status detects locally running solr in 14326ms
ok 48 status does not expose cli parameters to end user in 206ms
ok 49 listing out files in 2161ms
ok 50 copying files around in 6497ms
ok 51 upconfig in 2648ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 15m 57s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m20.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4905 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4905/

All tests passed

Build Log:
[...truncated 1254 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 23261ms
ok 43 post solr log into solr in 9141ms
ok 44 start solr with ssl in 20223ms
ok 45 start solr with ssl and auth in 23032ms
ok 46 SOLR-11740 check 'solr stop' connection in 28143ms
ok 47 status detects locally running solr in 14394ms
ok 48 status does not expose cli parameters to end user in 210ms
ok 49 listing out files in 2164ms
ok 50 copying files around in 6519ms
ok 51 upconfig in 2629ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 16m 1s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m28s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4904 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4904/

All tests passed

Build Log:
[...truncated 1258 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 23100ms
ok 43 post solr log into solr in 8760ms
ok 44 start solr with ssl in 20168ms
ok 45 start solr with ssl and auth in 22128ms
ok 46 SOLR-11740 check 'solr stop' connection in 28346ms
ok 47 status detects locally running solr in 14682ms
ok 48 status does not expose cli parameters to end user in 210ms
ok 49 listing out files in 2187ms
ok 50 copying files around in 6499ms
ok 51 upconfig in 2603ms

> Task :solr:packaging:integrationTests FAILED

FAILURE: Build failed with an exception.

* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 12m 56s
644 actionable tasks: 628 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 13m21s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4903 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4903/

3 tests failed.
FAILED:  org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess

Error Message:
java.lang.AssertionError: expected:<4> but was:<5>

Stack Trace:
java.lang.AssertionError: expected:<4> but was:<5>
	at __randomizedtesting.SeedInfo.seed([DFC17C2C341B463:C2E128C3A2C44930]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:633)
	at org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess(TestCoordinatorRole.java:573)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)


FAILED:  org.apache.solr.search.TestCoordinatorRole.classMethod

Error Message:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from SUITE scope at org.apache.solr.search.TestCoordinatorRole: 
   1) Thread[id=1707, name=TestCoordinatorRole-1100-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=1703, name=TestCoordinatorRole-1100-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=1711, name=TestCoordinatorRole-1100-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=1708, name=TestCoordinatorRole-1100-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=1710, name=TestCoordinatorRole-1100-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=1704, name=TestCoordinatorRole-1100-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=1709, name=TestCoordinatorRole-1100-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=1705, name=TestCoordinatorRole-1100-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=1716, name=TestCoordinatorRole-1100-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=1706, name=TestCoordinatorRole-1100-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from SUITE scope at org.apache.solr.search.TestCoordinatorRole: 
   1) Thread[id=1707, name=TestCoordinatorRole-1100-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=1703, name=TestCoordinatorRole-1100-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=1711, name=TestCoordinatorRole-1100-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=1708, name=TestCoordinatorRole-1100-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=1710, name=TestCoordinatorRole-1100-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=1704, name=TestCoordinatorRole-1100-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=1709, name=TestCoordinatorRole-1100-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=1705, name=TestCoordinatorRole-1100-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=1716, name=TestCoordinatorRole-1100-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=1706, name=TestCoordinatorRole-1100-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
	at __randomizedtesting.SeedInfo.seed([DFC17C2C341B463]:0)


FAILED:  org.apache.solr.search.TestCoordinatorRole.classMethod

Error Message:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=1710, name=TestCoordinatorRole-1100-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=1704, name=TestCoordinatorRole-1100-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=1707, name=TestCoordinatorRole-1100-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=1709, name=TestCoordinatorRole-1100-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=1705, name=TestCoordinatorRole-1100-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=1716, name=TestCoordinatorRole-1100-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=1703, name=TestCoordinatorRole-1100-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=1706, name=TestCoordinatorRole-1100-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=1711, name=TestCoordinatorRole-1100-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=1708, name=TestCoordinatorRole-1100-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=1710, name=TestCoordinatorRole-1100-thread-8, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   2) Thread[id=1704, name=TestCoordinatorRole-1100-thread-2, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   3) Thread[id=1707, name=TestCoordinatorRole-1100-thread-5, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   4) Thread[id=1709, name=TestCoordinatorRole-1100-thread-7, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   5) Thread[id=1705, name=TestCoordinatorRole-1100-thread-3, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   6) Thread[id=1716, name=TestCoordinatorRole-1100-thread-10, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   7) Thread[id=1703, name=TestCoordinatorRole-1100-thread-1, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   8) Thread[id=1706, name=TestCoordinatorRole-1100-thread-4, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
   9) Thread[id=1711, name=TestCoordinatorRole-1100-thread-9, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  10) Thread[id=1708, name=TestCoordinatorRole-1100-thread-6, state=WAITING, group=TGRP-TestCoordinatorRole]
        at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
        at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
        at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
        at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
        at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
	at __randomizedtesting.SeedInfo.seed([DFC17C2C341B463]:0)




Build Log:
[...truncated 9647 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22937ms
ok 43 post solr log into solr in 9411ms
ok 44 start solr with ssl in 19980ms
ok 45 start solr with ssl and auth in 22821ms
ok 46 SOLR-11740 check 'solr stop' connection in 28119ms
ok 47 status detects locally running solr in 14386ms
ok 48 status does not expose cli parameters to end user in 217ms
ok 49 listing out files in 2193ms
ok 50 copying files around in 6570ms
ok 51 upconfig in 2644ms

> Task :solr:packaging:integrationTests FAILED

ERROR: The following test(s) have failed:
  - org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestCoordinatorRole.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.TestCoordinatorRole.testConcurrentAccess" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=DFC17C2C341B463 -Ptests.file.encoding=US-ASCII

  - org.apache.solr.search.TestCoordinatorRole.classMethod (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestCoordinatorRole.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.TestCoordinatorRole.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=DFC17C2C341B463 -Ptests.file.encoding=US-ASCII

  - org.apache.solr.search.TestCoordinatorRole.classMethod (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.search.TestCoordinatorRole.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.search.TestCoordinatorRole.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=DFC17C2C341B463 -Ptests.file.encoding=US-ASCII


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:core:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/core/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 13m 10s
625 actionable tasks: 609 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 13m34.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Solr » Solr-Check-9.x - Build # 4902 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4902/

1 tests failed.
FAILED:  org.apache.solr.cloud.TestLeaderElectionZkExpiry.classMethod

Error Message:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestLeaderElectionZkExpiry: 
   1) Thread[id=323, name=OverseerExitThread, state=TIMED_WAITING, group=Overseer state updater.]
        at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
        at app//org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:161)
        at app//org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:82)
        at app//org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:401)
        at app//org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:103)
        at app//org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:315)
        at app//org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:395)
        at app//org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:2435)
        at app//org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:511)
        at app//org.apache.solr.cloud.Overseer$ClusterStateUpdater$$Lambda$1422/0x000000084070e040.run(Unknown Source)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.TestLeaderElectionZkExpiry: 
   1) Thread[id=323, name=OverseerExitThread, state=TIMED_WAITING, group=Overseer state updater.]
        at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
        at app//org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:161)
        at app//org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:82)
        at app//org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:401)
        at app//org.apache.solr.cloud.LeaderElector.checkIfIamLeader(LeaderElector.java:103)
        at app//org.apache.solr.cloud.LeaderElector.joinElection(LeaderElector.java:315)
        at app//org.apache.solr.cloud.LeaderElector.retryElection(LeaderElector.java:395)
        at app//org.apache.solr.cloud.ZkController.rejoinOverseerElection(ZkController.java:2435)
        at app//org.apache.solr.cloud.Overseer$ClusterStateUpdater.checkIfIamStillLeader(Overseer.java:511)
        at app//org.apache.solr.cloud.Overseer$ClusterStateUpdater$$Lambda$1422/0x000000084070e040.run(Unknown Source)
        at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
	at __randomizedtesting.SeedInfo.seed([C7309B22E7FEB55D]:0)




Build Log:
[...truncated 2193 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME. Only able to place 0 replicas.
ok 42 Random placement plugin using ENV in 22331ms
ok 43 post solr log into solr in 9475ms
ok 44 start solr with ssl in 19826ms
ok 45 start solr with ssl and auth in 22611ms
ok 46 SOLR-11740 check 'solr stop' connection in 28408ms
ok 47 status detects locally running solr in 14385ms
ok 48 status does not expose cli parameters to end user in 213ms
ok 49 listing out files in 2163ms
ok 50 copying files around in 6510ms
ok 51 upconfig in 2647ms

> Task :solr:packaging:integrationTests FAILED

ERROR: The following test(s) have failed:
  - org.apache.solr.cloud.TestLeaderElectionZkExpiry.classMethod (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.cloud.TestLeaderElectionZkExpiry.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.cloud.TestLeaderElectionZkExpiry.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=C7309B22E7FEB55D -Ptests.file.encoding=ISO-8859-1


FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:core:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/core/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 15m 54s
625 actionable tasks: 609 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 16m18s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1684) [Receiver=3.1.3]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any