[jira] [Created] (LUCENE-9469) Use RandomMockMergePolicy more often

["Simon Willnauer (Jira)" <ji...@apache.org>]

Simon Willnauer created LUCENE-9469:
---------------------------------------

             Summary: Use RandomMockMergePolicy more often
                 Key: LUCENE-9469
                 URL: https://issues.apache.org/jira/browse/LUCENE-9469
             Project: Lucene - Core
          Issue Type: Test
            Reporter: Simon Willnauer


During the work on LUCENE-8962 we had some discussions about testability of the feature and how well we can guarantee that it's tested. One argument was that we can rely to a certain extend on the randomization and enabling these features by swapping in RandomMockMergePolicy. 
I ran a test which throws an assertion error every time the feature is explicitly used which required MockRandomMergePolicy to be used. Unfortunately I had two entire test runs without any failure except of the tests that explicitly enabled it ie. the ones that I wrote for the feature. 
I think we are not using this MP often enough in our tests and I want to propose to use it way more frequently. It's certainly not a replacement for dedicated unit tests, I wrote a dedicated one for every random failure I found which should be common practice but it would be great to increase coverage by swapping in MockRandomMergePolicy more often. Maybe something as simple as this would do it:

{noformat}
--- a/lucene/test-framework/src/java/org/apache/lucene/util/LuceneTestCase.java
+++ b/lucene/test-framework/src/java/org/apache/lucene/util/LuceneTestCase.java
@@ -1059,7 +1059,7 @@ public abstract class LuceneTestCase extends Assert {
   }
 
   public static MergePolicy newMergePolicy(Random r, boolean includeMockMP) {
-    if (includeMockMP && rarely(r)) {
+    if (includeMockMP && r.nextBoolean()) {
       return new MockRandomMergePolicy(r);
     } else if (r.nextBoolean()) {
       return newTieredMergePolicy(r);
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org