[GitHub] [lucene-solr] dweiss commented on a change in pull request #2267: LUCENE-9707: Hunspell: check Lucene's implementation against Hunspel's test data

[GitBox <gi...@apache.org>]

dweiss commented on a change in pull request #2267:
URL: https://github.com/apache/lucene-solr/pull/2267#discussion_r567652699



##########
File path: lucene/analysis/common/src/test/org/apache/lucene/analysis/hunspell/SpellCheckerTest.java
##########
@@ -61,59 +61,74 @@ public void needAffixOnAffixes() throws Exception {
     doTest("needaffix5");
   }
 
+  @Test

Review comment:
       The test runner in Lucene will include any methods starting with "test" so you don't have to annotate them, unless you want to.

##########
File path: lucene/analysis/common/src/test/org/apache/lucene/analysis/hunspell/SpellCheckerTest.java
##########
@@ -61,59 +61,74 @@ public void needAffixOnAffixes() throws Exception {
     doTest("needaffix5");
   }
 
+  @Test
   public void testBreak() throws Exception {
     doTest("break");
   }
 
-  public void testBreakDefault() throws Exception {
+  @Test
+  public void breakDefault() throws Exception {
     doTest("breakdefault");
   }
 
-  public void testBreakOff() throws Exception {
+  @Test
+  public void breakOff() throws Exception {
     doTest("breakoff");
   }
 
-  public void testCompoundrule() throws Exception {
+  @Test
+  public void compoundrule() throws Exception {
     doTest("compoundrule");
   }
 
-  public void testCompoundrule2() throws Exception {
+  @Test
+  public void compoundrule2() throws Exception {
     doTest("compoundrule2");
   }
 
-  public void testCompoundrule3() throws Exception {
+  @Test
+  public void compoundrule3() throws Exception {
     doTest("compoundrule3");
   }
 
-  public void testCompoundrule4() throws Exception {
+  @Test
+  public void compoundrule4() throws Exception {
     doTest("compoundrule4");
   }
 
-  public void testCompoundrule5() throws Exception {
+  @Test
+  public void compoundrule5() throws Exception {
     doTest("compoundrule5");
   }
 
-  public void testCompoundrule6() throws Exception {
+  @Test
+  public void compoundrule6() throws Exception {
     doTest("compoundrule6");
   }
 
-  public void testCompoundrule7() throws Exception {
+  @Test
+  public void compoundrule7() throws Exception {
     doTest("compoundrule7");
   }
 
-  public void testCompoundrule8() throws Exception {
+  @Test
+  public void compoundrule8() throws Exception {
     doTest("compoundrule8");
   }
 
-  public void testGermanCompounding() throws Exception {
+  @Test
+  public void germanCompounding() throws Exception {
     doTest("germancompounding");
   }
 
   protected void doTest(String name) throws Exception {
-    InputStream affixStream =
-        Objects.requireNonNull(getClass().getResourceAsStream(name + ".aff"), name);
-    InputStream dictStream =
-        Objects.requireNonNull(getClass().getResourceAsStream(name + ".dic"), name);
+    checkSpellCheckerExpectations(

Review comment:
       Can you change it back to resource stream based implementations? Resource paths are not required to be on the default file system and then this code would fail with an odd exception. Pass the name of the resource and derive resource streams from that.

##########
File path: lucene/analysis/common/src/test/org/apache/lucene/analysis/hunspell/TestsFromOriginalHunspellRepository.java
##########
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.lucene.analysis.hunspell;
+
+import java.io.IOException;
+import java.nio.file.DirectoryStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.text.ParseException;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Set;
+import java.util.TreeSet;
+import java.util.stream.Collectors;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.junit.runners.Parameterized;
+
+/**
+ * Same as {@link SpellCheckerTest}, but checks all Hunspell's test data. The path to the checked
+ * out Hunspell repository should be in {@code -Dhunspell.repo.path=...} system property.
+ */
+@RunWith(Parameterized.class)

Review comment:
       Please run with the default runner (extend RandomizedTest or, better, LuceneTestCase) - it adds a number of extra checks to make sure no interference between tests, proper reporting, etc. If you wish to parameterize via constructor, you still can - an example is here:
   
   https://github.com/randomizedtesting/randomizedtesting/blob/master/examples/maven/src/main/java/com/carrotsearch/examples/randomizedrunner/Test007ParameterizedTests.java

##########
File path: lucene/analysis/common/src/test/org/apache/lucene/analysis/hunspell/SpellCheckerTest.java
##########
@@ -125,17 +140,17 @@ protected void doTest(String name) throws Exception {
       IOUtils.closeWhileHandlingException(dictStream);
     }
 
-    URL good = StemmerTestBase.class.getResource(name + ".good");
-    if (good != null) {
-      for (String word : Files.readAllLines(Path.of(good.toURI()))) {

Review comment:
       I see this was incorrect before as well. Worth correcting though.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org