[GitHub] [lucene] tang-hi commented on pull request #12254: add ConcurrentOnHeapHnswGraph and Builder

["tang-hi (via GitHub)" <gi...@apache.org>]

tang-hi commented on PR #12254:
URL: https://github.com/apache/lucene/pull/12254#issuecomment-1529914311

   > I've made a couple optimizations and merged to main.
   > 
   > With the optimizations it's still about 38% slower than the non-concurrent code (including the HashMap optimization for the non-concurrent that went in). Nothing stands out in the profiler as a problem anymore; I guess that's what you get replacing HashMap with ConcurrentHashMap and more especially replacing ArrayList with ConcurrentHashMap in L0.
   > 
   > I still do not know how to fix the ram usage estimates; any suggestions?
   
   I believe the issue is related to the Java Platform Module System, which was introduced in Java 9. You can modify the following section in the [gradlew script](https://github.com/apache/lucene/blob/3c163745bb07aed51b52878de0666f1405696147/gradlew#L221) to avoid the exception:
   https://github.com/apache/lucene/blob/3c163745bb07aed51b52878de0666f1405696147/gradlew#L221
   
   **modified version**
   ```shell
   JVM_ARGS=" -Ptests.jvmargs=\"--add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.util.concurrent.atomic=ALL-UNNAMED\""
   
   # Collect all arguments for the java command, following the shell quoting and substitution rules
   
   eval set -- $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS" "$JVM_ARGS"
   ```
   After making this change, the exception should no longer be thrown. However, the test may still fail due to an AssertionError, as shown below:
   
   ```
     java.lang.AssertionError: expected:<1185336.0> but was:<486483.0>
      >         at __randomizedtesting.SeedInfo.seed([A41F62241729FCF:CF0592627AF94878]:0)
      >         at org.junit.Assert.fail(Assert.java:89)
      >         at org.junit.Assert.failNotEquals(Assert.java:835)
      >         at org.junit.Assert.assertEquals(Assert.java:555)
      >         at org.junit.Assert.assertEquals(Assert.java:685)
      >         at org.apache.lucene.util.hnsw.HnswGraphTestCase.testRamUsageEstimate(HnswGraphTestCase.java:789)
      >         at org.apache.lucene.util.hnsw.TestHnswFloatVectorGraph.testRamUsageEstimate(TestHnswFloatVectorGraph.java:37)
      >         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      >         at java.base/jdk.internal.reflect.NativeMethodAccesso
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org