You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@solr.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2023/06/09 18:28:45 UTC

[JENKINS] Solr » Solr-Check-9.x - Build # 4567 - Still Failing!

Build: https://ci-builds.apache.org/job/Solr/job/Solr-Check-9.x/4567/

2 tests failed.
FAILED:  org.apache.solr.util.TestCircuitBreaker.testResponseWithCBTiming

Error Message:
java.lang.RuntimeException: Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([998DCBD6441C0336:71516634B7F41916]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:1024)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:982)
	at org.apache.solr.util.TestCircuitBreaker.testResponseWithCBTiming(TestCircuitBreaker.java:220)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.solr.common.SolrException: Circuit Breakers tripped CPU Circuit Breaker triggered as seen CPU usage is above allowed threshold.Seen CPU usage 81.23 and allocated threshold 75.0

	at app//org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:375)
	at app//org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:224)
	at app//org.apache.solr.core.SolrCore.execute(SolrCore.java:2893)
	at app//org.apache.solr.util.TestHarness.query(TestHarness.java:348)
	at app//org.apache.solr.util.TestHarness.query(TestHarness.java:328)
	at app//org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:996)
	... 49 more


FAILED:  org.apache.solr.ltr.feature.TestFeatureExtractionFromMultipleSegments.classMethod

Error Message:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([998DCBD6441C0336]:0)




Build Log:
[...truncated 1652 lines...]
# ERROR: Error from server at http://localhost:8983/solr/admin/collections?action=CREATE&name=COLL_NAME&collection.configName=COLL_NAME&numShards=1&nrtReplicas=1&wt=javabin&version=2: org.apache.solr.cluster.placement.PlacementException: Not enough eligible nodes to place 1 replica(s) of type NRT for shard shard1 of collection COLL_NAME
ok 40 Random placement plugin using ENV in 22835ms
ok 41 post solr log into solr in 9476ms
ok 42 start solr with ssl in 20052ms
ok 43 start solr with ssl and auth in 22706ms
ok 44 SOLR-11740 check 'solr stop' connection in 28535ms
ok 45 status detects locally running solr in 14428ms
ok 46 status does not expose cli parameters to end user in 213ms
ok 47 listing out files in 2179ms
ok 48 copying files around in 6592ms
ok 49 upconfig in 2732ms

> Task :solr:packaging:integrationTests FAILED

> Task :solr:modules:ltr:test

org.apache.solr.ltr.feature.TestFeatureExtractionFromMultipleSegments > classMethod FAILED
    java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
        at __randomizedtesting.SeedInfo.seed([998DCBD6441C0336]:0)

org.apache.solr.ltr.feature.TestFeatureExtractionFromMultipleSegments > test suite's output saved to /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.feature.TestFeatureExtractionFromMultipleSegments.txt, copied below:
  1> 16:27:32.155 [SUITE-TestFeatureExtractionFromMultipleSegments-seed#[998DCBD6441C0336]-worker] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
  2> يونيو ١٠, ٢٠٢٣ ١:٢٧:٣٠ ص com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
  2> WARNING: Suite execution timed out: org.apache.solr.ltr.feature.TestFeatureExtractionFromMultipleSegments
  2> ==== jstack at approximately timeout time ====
  2> "Test worker" ID=1 WAITING on com.carrotsearch.randomizedtesting.RandomizedRunner$2@16484981
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on com.carrotsearch.randomizedtesting.RandomizedRunner$2@16484981
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1300)
  2> 	at java.base@11.0.16/java.lang.Thread.join(Thread.java:1375)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:647)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:504)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:108)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:57)
  2> 	at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:39)
  2> 	at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
  2> 	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:52)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
  2> 	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
  2> 	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
  2> 	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
  2> 	at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
  2> 	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
  2> 	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
  2> 	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
  2> 	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
  2> 
  2> "Reference Handler" ID=2 RUNNABLE
  2> 	at java.base@11.0.16/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
  2> 	at java.base@11.0.16/java.lang.ref.Reference.processPendingReferences(Reference.java:241)
  2> 	at java.base@11.0.16/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)
  2> 
  2> "Finalizer" ID=3 WAITING on java.lang.ref.ReferenceQueue$Lock@477f555b
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.ref.ReferenceQueue$Lock@477f555b
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
  2> 	at java.base@11.0.16/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)
  2> 
  2> "Signal Dispatcher" ID=4 RUNNABLE
  2> 
  2> "Common-Cleaner" ID=9 TIMED_WAITING on java.lang.ref.ReferenceQueue$Lock@2de2a191
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- timed waiting on java.lang.ref.ReferenceQueue$Lock@2de2a191
  2> 	at java.base@11.0.16/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
  2> 	at java.base@11.0.16/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	at java.base@11.0.16/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:161)
  2> 
  2> "/127.0.0.1:47254 to /127.0.0.1:43669 workers" ID=11 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@56299525
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@56299525
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:403)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@1d9b7cce
  2> 
  2> "/127.0.0.1:47254 to /127.0.0.1:43669 workers Thread 2" ID=12 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@37c5bed1
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@37c5bed1
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at org.gradle.internal.remote.internal.hub.queue.EndPointQueue.take(EndPointQueue.java:49)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionDispatch.run(MessageHub.java:322)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@7d8995e
  2> 
  2> "/127.0.0.1:47254 to /127.0.0.1:43669 workers Thread 3" ID=13 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@aadf0ff
  2> 	- locked sun.nio.ch.EPollSelectorImpl@32fb3612
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at org.gradle.internal.remote.internal.inet.SocketConnection$SocketInputStream.read(SocketConnection.java:185)
  2> 	at com.esotericsoftware.kryo.io.Input.fill(Input.java:146)
  2> 	at com.esotericsoftware.kryo.io.Input.require(Input.java:178)
  2> 	at com.esotericsoftware.kryo.io.Input.readByte(Input.java:295)
  2> 	at org.gradle.internal.serialize.kryo.KryoBackedDecoder.readByte(KryoBackedDecoder.java:88)
  2> 	at org.gradle.internal.remote.internal.hub.InterHubMessageSerializer$MessageReader.read(InterHubMessageSerializer.java:64)
  2> 	at org.gradle.internal.remote.internal.hub.InterHubMessageSerializer$MessageReader.read(InterHubMessageSerializer.java:52)
  2> 	at org.gradle.internal.remote.internal.inet.SocketConnection.receive(SocketConnection.java:81)
  2> 	at org.gradle.internal.remote.internal.hub.MessageHub$ConnectionReceive.run(MessageHub.java:270)
  2> 	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
  2> 	at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@6aeb35e6
  2> 
  2> "SUITE-TestFeatureExtractionFromMultipleSegments-seed#[998DCBD6441C0336]" ID=14 RUNNABLE
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpThreads0(Native Method)
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:521)
  2> 	at java.management@11.0.16/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:509)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:708)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:59)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:437)
  2> 	- locked java.lang.Object@28e11dd5
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:716)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:637)
  2> 
  2> "SUITE-TestFeatureExtractionFromMultipleSegments-seed#[998DCBD6441C0336]-worker" ID=15 WAITING on java.util.concurrent.CountDownLatch$Sync@fe4310
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.CountDownLatch$Sync@fe4310
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
  2> 	at java.base@11.0.16/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
  2> 	at app//org.apache.solr.servlet.CoreContainerProvider$ContextInitializationKey.waitForReadyService(CoreContainerProvider.java:510)
  2> 	at app//org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:549)
  2> 	at app//org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
  2> 	at app//org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472)
  2> 	at app//org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992)
  2> 	at app//org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427)
  2> 	- locked org.apache.solr.embedded.JettySolrRunner$1@32481d2e
  2> 	at app//org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253)
  2> 	at app//org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616)
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554)
  2> 	- locked org.apache.solr.embedded.JettySolrRunner@57072990
  2> 	at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525)
  2> 	at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126)
  2> 	at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86)
  2> 	at app//org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57)
  2> 	at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181)
  2> 	at app//org.apache.solr.ltr.feature.TestFeatureExtractionFromMultipleSegments.before(TestFeatureExtractionFromMultipleSegments.java:42)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2> 	at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2> 	at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2> 	at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:886)
  2> 	at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  2> 	at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2> 	at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2> 	at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  2> 	at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2> 	at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  2> 	at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$$Lambda$177/0x00000008401edc40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@3c27dec4
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@1c087320
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@6e5bace2
  2> 
  2> "qtp793780485-17" ID=17 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@2ad7c35
  2> 	- locked sun.nio.ch.EPollSelectorImpl@5de39e33
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$259/0x000000084034fc40.run(Unknown Source)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp793780485-18-acceptor-٠@4df86c36-ServerConnector@34b0c3a1{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:٤٥٩٣٩}" ID=18 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
  2> 	at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
  2> 	at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
  2> 	at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.locks.ReentrantLock$NonfairSync@6e6c05d1
  2> 
  2> "qtp793780485-19" ID=19 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf82dcf
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf82dcf
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp793780485-20" ID=20 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf82dcf
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf82dcf
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp793780485-21" ID=21 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf82dcf
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf82dcf
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp793780485-22" ID=22 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf82dcf
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf82dcf
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp793780485-23" ID=23 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf82dcf
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf82dcf
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "qtp793780485-24" ID=24 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf82dcf
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@1cf82dcf
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2> 	at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "Session-HouseKeeper-6c44bc55-1" ID=25 TIMED_WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@75ddb6b1
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- timed waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@75ddb6b1
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2> 	at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
  2> 	at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "httpShardExecutor-6-thread-1" ID=26 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@340eac3f
  2> 	- locked sun.nio.ch.EPollSelectorImpl@3a4275e2
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$259/0x000000084034fc40.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$348/0x0000000840446c40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@2897a50e
  2> 
  2> "httpShardExecutor-6-thread-2" ID=27 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@6709a130
  2> 	- locked sun.nio.ch.EPollSelectorImpl@7ff9fb5e
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$259/0x000000084034fc40.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$348/0x0000000840446c40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@7ff7f541
  2> 
  2> "Connection evictor" ID=28 TIMED_WAITING
  2> 	at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
  2> 	at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "Connection evictor" ID=29 TIMED_WAITING
  2> 	at java.base@11.0.16/java.lang.Thread.sleep(Native Method)
  2> 	at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> "h2sc-8-thread-1" ID=30 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@3e57dfca
  2> 	- locked sun.nio.ch.EPollSelectorImpl@17135087
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$259/0x000000084034fc40.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$348/0x0000000840446c40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@26711fb0
  2> 
  2> "h2sc-8-thread-2" ID=31 RUNNABLE (in native code)
  2> 	at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2> 	at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2> 	- locked sun.nio.ch.Util$2@3d396374
  2> 	- locked sun.nio.ch.EPollSelectorImpl@d2a5c86
  2> 	at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2> 	at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2> 	at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$259/0x000000084034fc40.run(Unknown Source)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2> 	at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$348/0x0000000840446c40.run(Unknown Source)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 	Locked synchronizers:
  2> 	- java.util.concurrent.ThreadPoolExecutor$Worker@4f9e5f15
  2> 
  2> "CloserThread" ID=33 WAITING on java.lang.Object@d036b66
  2> 	at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2> 	- waiting on java.lang.Object@d036b66
  2> 	at java.base@11.0.16/java.lang.Object.wait(Object.java:328)
  2> 	at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2554)
  2> 
  2> "ForkJoinPool.commonPool-worker-3" ID=36 WAITING on java.util.concurrent.ForkJoinPool@999529
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.ForkJoinPool@999529
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
  2> 	at java.base@11.0.16/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
  2> 
  2> "searcherExecutor-13-thread-1" ID=37 WAITING on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@76cd748d
  2> 	at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2> 	- waiting on java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@76cd748d
  2> 	at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2> 	at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2> 	at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2> 	at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2> 	at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> 
  2> ^^==============================================
  2> 
  2> يونيو ١٠, ٢٠٢٣ ١:٢٧:٣٠ ص com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> WARNING: Will linger awaiting termination of 16 leaked thread(s).
  2> يونيو ١٠, ٢٠٢٣ ١:٢٧:٣١ ص com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
  2> SEVERE: 16 threads leaked from SUITE scope at org.apache.solr.ltr.feature.TestFeatureExtractionFromMultipleSegments: 
  2>    1) Thread[id=19, name=qtp793780485-19, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    2) Thread[id=24, name=qtp793780485-24, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    3) Thread[id=21, name=qtp793780485-21, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    4) Thread[id=18, name=qtp793780485-18-acceptor-٠@4df86c36-ServerConnector@34b0c3a1{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:٤٥٩٣٩}, state=RUNNABLE, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:533)
  2>         at java.base@11.0.16/sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:285)
  2>         at app//org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:409)
  2>         at app//org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:748)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    5) Thread[id=17, name=qtp793780485-17, state=RUNNABLE, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$259/0x000000084034fc40.run(Unknown Source)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:969)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.doRunJob(QueuedThreadPool.java:1194)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1149)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    6) Thread[id=20, name=qtp793780485-20, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    7) Thread[id=26, name=httpShardExecutor-6-thread-1, state=RUNNABLE, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$259/0x000000084034fc40.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$348/0x0000000840446c40.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    8) Thread[id=27, name=httpShardExecutor-6-thread-2, state=RUNNABLE, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$259/0x000000084034fc40.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$348/0x0000000840446c40.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>    9) Thread[id=37, name=searcherExecutor-13-thread-1, state=WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2081)
  2>         at java.base@11.0.16/java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:433)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   10) Thread[id=22, name=qtp793780485-22, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   11) Thread[id=30, name=h2sc-8-thread-1, state=RUNNABLE, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$259/0x000000084034fc40.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$348/0x0000000840446c40.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   12) Thread[id=23, name=qtp793780485-23, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at app//org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:219)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:1124)
  2>         at app//org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1141)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   13) Thread[id=31, name=h2sc-8-thread-2, state=RUNNABLE, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/sun.nio.ch.EPoll.wait(Native Method)
  2>         at java.base@11.0.16/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:120)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:124)
  2>         at java.base@11.0.16/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:180)
  2>         at app//org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:187)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:604)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:541)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produceTask(AdaptiveExecutionStrategy.java:450)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.tryProduce(AdaptiveExecutionStrategy.java:243)
  2>         at app//org.eclipse.jetty.util.thread.strategy.AdaptiveExecutionStrategy.produce(AdaptiveExecutionStrategy.java:194)
  2>         at app//org.eclipse.jetty.io.ManagedSelector$$Lambda$259/0x000000084034fc40.run(Unknown Source)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:289)
  2>         at app//org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$348/0x0000000840446c40.run(Unknown Source)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   14) Thread[id=25, name=Session-HouseKeeper-6c44bc55-1, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
  2>         at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
  2>         at java.base@11.0.16/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
  2>         at java.base@11.0.16/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2>   15) Thread[id=33, name=CloserThread, state=WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/java.lang.Object.wait(Native Method)
  2>         at java.base@11.0.16/java.lang.Object.wait(Object.java:328)
  2>         at app//org.apache.solr.core.CloserThread.run(CoreContainer.java:2554)
  2>   16) Thread[id=15, name=SUITE-TestFeatureExtractionFromMultipleSegments-seed#[998DCBD6441C0336]-worker, state=WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>         at java.base@11.0.16/jdk.internal.misc.Unsafe.park(Native Method)
  2>         at java.base@11.0.16/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:885)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1039)
  2>         at java.base@11.0.16/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1345)
  2>         at java.base@11.0.16/java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
  2>         at app//org.apache.solr.servlet.CoreContainerProvider$ContextInitializationKey.waitForReadyService(CoreContainerProvider.java:510)
  2>         at app//org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:549)
  2>         at app//org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147)
  2>         at app//org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472)
  2>         at app//org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992)
  2>         at app//org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480)
  2>         at app//org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427)
  2>         at app//org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253)
  2>         at app//org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554)
  2>         at app//org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525)
  2>         at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126)
  2>         at app//org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86)
  2>         at app//org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57)
  2>         at app//org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181)
  2>         at app//org.apache.solr.ltr.feature.TestFeatureExtractionFromMultipleSegments.before(TestFeatureExtractionFromMultipleSegments.java:42)
  2>         at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  2>         at java.base@11.0.16/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  2>         at java.base@11.0.16/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  2>         at java.base@11.0.16/java.lang.reflect.Method.invoke(Method.java:566)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:886)
  2>         at app//com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  2>         at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2>         at app//com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  2>         at app//org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  2>         at app//org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  2>         at app//org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  2>         at app//org.junit.rules.RunRules.evaluate(RunRules.java:20)
  2>         at app//com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  2>         at app//com.carrotsearch.randomizedtesting.ThreadLeakControl$$Lambda$177/0x00000008401edc40.run(Unknown Source)
  2>         at java.base@11.0.16/java.lang.Thread.run(Thread.java:829)
  2> يونيو ١٠, ٢٠٢٣ ١:٢٧:٣١ ص com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFO: Starting to interrupt leaked threads:
  2>    1) Thread[id=19, name=qtp793780485-19, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>    2) Thread[id=24, name=qtp793780485-24, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>    3) Thread[id=21, name=qtp793780485-21, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>    4) Thread[id=18, name=qtp793780485-18-acceptor-٠@4df86c36-ServerConnector@34b0c3a1{ssl, (ssl, alpn, http/1.1, h2)}{127.0.0.1:٤٥٩٣٩}, state=RUNNABLE, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>    5) Thread[id=17, name=qtp793780485-17, state=RUNNABLE, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>    6) Thread[id=20, name=qtp793780485-20, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>    7) Thread[id=26, name=httpShardExecutor-6-thread-1, state=RUNNABLE, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>    8) Thread[id=27, name=httpShardExecutor-6-thread-2, state=RUNNABLE, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>    9) Thread[id=37, name=searcherExecutor-13-thread-1, state=WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>   10) Thread[id=22, name=qtp793780485-22, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>   11) Thread[id=30, name=h2sc-8-thread-1, state=RUNNABLE, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>   12) Thread[id=23, name=qtp793780485-23, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>   13) Thread[id=31, name=h2sc-8-thread-2, state=RUNNABLE, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>   14) Thread[id=25, name=Session-HouseKeeper-6c44bc55-1, state=TIMED_WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>   15) Thread[id=33, name=CloserThread, state=WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  2>   16) Thread[id=15, name=SUITE-TestFeatureExtractionFromMultipleSegments-seed#[998DCBD6441C0336]-worker, state=WAITING, group=TGRP-TestFeatureExtractionFromMultipleSegments]
  1> 18:27:31.365 [SUITE-TestFeatureExtractionFromMultipleSegments-seed#[998DCBD6441C0336]-worker] ERROR org.apache.solr.servlet.SolrDispatchFilter - Could not start Dispatch Filter.
  1> org.apache.solr.common.SolrException: Interrupted while obtaining reference to CoreService
  1> 	at org.apache.solr.servlet.CoreContainerProvider$ServiceHolder.getService(CoreContainerProvider.java:556) ~[solr-core-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT 12177c9197187c532fb6bff595510064e17a7af9 [snapshot build, details omitted]]
  1> 	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:147) [solr-core-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT 12177c9197187c532fb6bff595510064e17a7af9 [snapshot build, details omitted]]
  1> 	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:133) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.initializeHolders(ServletHandler.java:774) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.setFilters(ServletHandler.java:1472) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:992) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:480) [jetty-servlet-10.0.15.jar:10.0.15]
  1> 	at org.apache.solr.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:427) [solr-test-framework-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT 12177c9197187c532fb6bff595510064e17a7af9 [snapshot build, details omitted]]
  1> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:253) [jetty-util-10.0.15.jar:10.0.15]
  1> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:94) [jetty-util-10.0.15.jar:10.0.15]
  1> 	at org.apache.solr.embedded.JettySolrRunner.retryOnPortBindFailure(JettySolrRunner.java:616) [solr-test-framework-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT 12177c9197187c532fb6bff595510064e17a7af9 [snapshot build, details omitted]]
  1> 	at org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:554) [solr-test-framework-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT 12177c9197187c532fb6bff595510064e17a7af9 [snapshot build, details omitted]]
  1> 	at org.apache.solr.embedded.JettySolrRunner.start(JettySolrRunner.java:525) [solr-test-framework-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT 12177c9197187c532fb6bff595510064e17a7af9 [snapshot build, details omitted]]
  1> 	at org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:126) [solr-test-framework-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT 12177c9197187c532fb6bff595510064e17a7af9 [snapshot build, details omitted]]
  1> 	at org.apache.solr.SolrJettyTestBase.createAndStartJetty(SolrJettyTestBase.java:86) [solr-test-framework-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT 12177c9197187c532fb6bff595510064e17a7af9 [snapshot build, details omitted]]
  1> 	at org.apache.solr.util.RestTestBase.createJettyAndHarness(RestTestBase.java:57) [solr-test-framework-9.3.0-SNAPSHOT.jar:9.3.0-SNAPSHOT 12177c9197187c532fb6bff595510064e17a7af9 [snapshot build, details omitted]]
  1> 	at org.apache.solr.ltr.TestRerankBase.setuptest(TestRerankBase.java:181) [test/:?]
  1> 	at org.apache.solr.ltr.feature.TestFeatureExtractionFromMultipleSegments.before(TestFeatureExtractionFromMultipleSegments.java:42) [test/:?]
  1> 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
  1> 	at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
  1> 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
  1> 	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:886) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.5.0.jar:9.5.0 13803aa6ea7fee91f798cfeded4296182ac43a21 - 2023-01-25 16:44:59]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.5.0.jar:9.5.0 13803aa6ea7fee91f798cfeded4296182ac43a21 - 2023-01-25 16:44:59]
  1> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:80) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.5.0.jar:9.5.0 13803aa6ea7fee91f798cfeded4296182ac43a21 - 2023-01-25 16:44:59]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) [lucene-test-framework-9.5.0.jar:9.5.0 13803aa6ea7fee91f798cfeded4296182ac43a21 - 2023-01-25 16:44:59]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) [lucene-test-framework-9.5.0.jar:9.5.0 13803aa6ea7fee91f798cfeded4296182ac43a21 - 2023-01-25 16:44:59]
  1> 	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) [lucene-test-framework-9.5.0.jar:9.5.0 13803aa6ea7fee91f798cfeded4296182ac43a21 - 2023-01-25 16:44:59]
  1> 	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) [lucene-test-framework-9.5.0.jar:9.5.0 13803aa6ea7fee91f798cfeded4296182ac43a21 - 2023-01-25 16:44:59]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) [lucene-test-framework-9.5.0.jar:9.5.0 13803aa6ea7fee91f798cfeded4296182ac43a21 - 2023-01-25 16:44:59]
  1> 	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) [lucene-test-framework-9.5.0.jar:9.5.0 13803aa6ea7fee91f798cfeded4296182ac43a21 - 2023-01-25 16:44:59]
  1> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.2.jar:4.13.2]
  1> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) [randomizedtesting-runner-2.8.1.jar:?]
  1> 	at java.lang.Thread.run(Thread.java:829) [?:?]
  2> Jun 09, 2023 6:27:32 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
  2> INFO: All leaked threads terminated.
   >     java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   >         at __randomizedtesting.SeedInfo.seed([998DCBD6441C0336]:0)
  2> NOTE: test params are: codec=Asserting(Lucene95), sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=ar-KW, timezone=Asia/Phnom_Penh
  2> NOTE: Linux 5.4.0-100-generic amd64/Oracle Corporation 11.0.16 (64-bit)/cpus=1,threads=1,free=257907208,total=324534272
  2> NOTE: All tests run in this JVM: [TestFeatureExtractionFromMultipleSegments]
  2> NOTE: reproduce with: gradlew test --tests TestFeatureExtractionFromMultipleSegments -Dtests.seed=998DCBD6441C0336 -Dtests.locale=ar-KW -Dtests.timezone=Asia/Phnom_Penh -Dtests.asserts=true -Dtests.file.encoding=UTF-8

:solr:modules:ltr:test (FAILURE): 186 test(s), 1 failure(s)

186 tests completed, 1 failed

> Task :solr:modules:ltr:test FAILED
> Task :solr:modules:ltr:wipeTaskTemp

ERROR: The following test(s) have failed:
  - org.apache.solr.ltr.feature.TestFeatureExtractionFromMultipleSegments.classMethod (:solr:modules:ltr)
    Test output: /tmp/src/solr/solr/modules/ltr/build/test-results/test/outputs/OUTPUT-org.apache.solr.ltr.feature.TestFeatureExtractionFromMultipleSegments.txt
    Reproduce with: gradlew :solr:modules:ltr:test --tests "org.apache.solr.ltr.feature.TestFeatureExtractionFromMultipleSegments.classMethod" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=998DCBD6441C0336 -Ptests.file.encoding=US-ASCII

  - org.apache.solr.util.TestCircuitBreaker.testResponseWithCBTiming (:solr:core)
    Test output: /tmp/src/solr/solr/core/build/test-results/test/outputs/OUTPUT-org.apache.solr.util.TestCircuitBreaker.txt
    Reproduce with: gradlew :solr:core:test --tests "org.apache.solr.util.TestCircuitBreaker.testResponseWithCBTiming" -Ptests.jvms=96 "-Ptests.jvmargs=-XX:TieredStopAtLevel=1 -XX:+UseParallelGC -XX:ActiveProcessorCount=1 -XX:ReservedCodeCacheSize=120m" -Ptests.seed=998DCBD6441C0336 -Ptests.file.encoding=US-ASCII


FAILURE: Build completed with 3 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:core:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/core/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

2: Task failed with an exception.
-----------
* Where:
Build file '/tmp/src/solr/solr/packaging/build.gradle' line: 285

* What went wrong:
Execution failed for task ':solr:packaging:integrationTests'.
> Process 'command '/tmp/src/solr/.gradle/node/packaging/node_modules/bats/bin/bats'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':solr:modules:ltr:test'.
> There were failing tests. See the results at: file:///tmp/src/solr/solr/modules/ltr/build/test-results/test/

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================

* Get more help at https://help.gradle.org

BUILD FAILED in 2h 2m 49s
625 actionable tasks: 609 executed, 16 up-to-date
[?12l[?25h
------------------------------------------------------------------------
Build Failed: returned 1

Total time: 2h3m14.5s
------------------------------------------------------------------------
+ status=1
+ ./crave pull '**/build/**/test/TEST-*.xml' '**/*.events' 'heapdumps/**' '**/hs_err_pid*'
Error: rsync: [sender] change_dir "/tmp/src/solr/heapdumps" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1675) [Receiver=3.1.2]
rsync: [Receiver] write error: Broken pipe (32)

+ exit 1
Build step 'Execute shell' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any