You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "aoli-al (via GitHub)" <gi...@apache.org> on 2024/07/07 17:01:39 UTC

[I] Flaky Test in TestMergeSchedulerExternal#testSubclassConcurrentMergeScheduler [lucene]

aoli-al opened a new issue, #13547:
URL: https://github.com/apache/lucene/issues/13547

   ### Description
   
   The test `testSubclassConcurrentMergeScheduler` is flaky due to concurrency issues and produces following assertion failure: 
   
   ```
   java.lang.AssertionError
   	at __randomizedtesting.SeedInfo.seed([8CC0C37B9C028773:B417ED69822FD77]:0)
   	at org.junit.Assert.fail(Assert.java:87)
   	at org.junit.Assert.assertTrue(Assert.java:42)
   	at org.junit.Assert.assertTrue(Assert.java:53)
   	at org.apache.lucene.TestMergeSchedulerExternal.testSubclassConcurrentMergeScheduler(TestMergeSchedulerExternal.java:161)
   	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
   	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
   	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
   	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
   	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
   	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
   	at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
   	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
   	at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
   	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
   	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
   	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
   	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
   	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
   	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
   	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
   	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
   	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
   	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
   	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
   	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   	at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
   	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   	at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   	at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
   	at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
   	at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
   	at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
   	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
   	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
   	at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
   	at java.base/java.lang.Thread.run(Thread.java:1583)
   ```
   
   The root cause is that the injected `IOException` in `MyMergeThread` will close IndexWriter https://github.com/apache/lucene/blob/62e08f5f4b76b84ed4109d8493d1dac2048d3350/lucene/core/src/java/org/apache/lucene/index/IndexWriter.java#L4777 . 
   
   If this code is executed before `((MyMergeScheduler) writer.getConfig().getMergeScheduler()).sync();` at line https://github.com/apache/lucene/blob/62e08f5f4b76b84ed4109d8493d1dac2048d3350/lucene/core/src/test/org/apache/lucene/TestMergeSchedulerExternal.java#L145 The testing thread will not block. And the `assertTrue(excCalled);` check might be called before the assignment at line https://github.com/apache/lucene/blob/main/lucene/core/src/test/org/apache/lucene/TestMergeSchedulerExternal.java#L77
   
   This might also be the root cause of #11683 and #11653 
   
   You can make the test more reproducible by applying the following patch to the `TestMergeSchedulerExternal.java`
   
   ```
   diff --git a/lucene/core/src/test/org/apache/lucene/TestMergeSchedulerExternal.java b/lucene/core/src/test/org/apache/lucene/TestMergeSchedulerExternal.java
   index 506a76f9989..3510e110659 100644
   --- a/lucene/core/src/test/org/apache/lucene/TestMergeSchedulerExternal.java
   +++ b/lucene/core/src/test/org/apache/lucene/TestMergeSchedulerExternal.java
   @@ -74,7 +74,11 @@ public class TestMergeSchedulerExternal extends LuceneTestCase {
    
        @Override
        protected void handleMergeException(Throwable t) {
   -      excCalled = true;
   +        try {
   +            Thread.sleep(10000);
   +        } catch (InterruptedException e) {
   +        }
   +        excCalled = true;
          if (infoStream.isEnabled("IW")) {
            infoStream.message("IW", "TEST: now handleMergeException");
          }
   @@ -141,6 +145,11 @@ public class TestMergeSchedulerExternal extends LuceneTestCase {
          // OK
        }
    
   +    try {
   +      Thread.sleep(100);
   +    } catch (InterruptedException ie) {
   +    }
   +
        try {
          ((MyMergeScheduler) writer.getConfig().getMergeScheduler()).sync();
        } catch (
   ```
   
   
   The complete test output: https://pastebin.com/Re8ijFcT
   
   
   
   ### Version and environment details
   
   My lucene commit: 33a4c1d8ef999902dacedde9c7f04a3c7e2e78c9
   JDK version: openjdk 21.0.3 2024-04-16
   OpenJDK Runtime Environment Homebrew (build 21.0.3)
   OpenJDK 64-Bit Server VM Homebrew (build 21.0.3, mixed mode, sharing)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] Flaky Test in TestMergeSchedulerExternal#testSubclassConcurrentMergeScheduler [lucene]

Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #13547:
URL: https://github.com/apache/lucene/issues/13547#issuecomment-2836236473

   Thank you again, @aoli-al, this is a great contribution and not an easy test bug to track down.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] Flaky Test in TestMergeSchedulerExternal#testSubclassConcurrentMergeScheduler [lucene]

Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss commented on issue #13547:
URL: https://github.com/apache/lucene/issues/13547#issuecomment-2835018138

   Your analysis is absolutely correct, @aoli-al ! I'd like to apologize for the delay, this slipped from the radar somehow. I've filed a pull request that bypasses getConfig() and sync's on the merge threads (this way those test variables are in correct state).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] Flaky Test in TestMergeSchedulerExternal#testSubclassConcurrentMergeScheduler [lucene]

Posted by "dweiss (via GitHub)" <gi...@apache.org>.
dweiss closed issue #13547: Flaky Test in TestMergeSchedulerExternal#testSubclassConcurrentMergeScheduler
URL: https://github.com/apache/lucene/issues/13547


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org


Re: [I] Flaky Test in TestMergeSchedulerExternal#testSubclassConcurrentMergeScheduler [lucene]

Posted by "aoli-al (via GitHub)" <gi...@apache.org>.
aoli-al commented on issue #13547:
URL: https://github.com/apache/lucene/issues/13547#issuecomment-2302212349

   Please use this fork to reproduce the failure: https://github.com/aoli-al/lucene/tree/LUCENE-13547
   Command: `./gradlew test --tests "*testSubclassConcurrentMergeScheduler*"`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org