[GitHub] [lucene] sherman commented on pull request #912: MR-JAR rewrite of MMapDirectory with JDK-19 preview Panama APIs (>= JDK-19-ea+23)

[GitBox <gi...@apache.org>]

sherman commented on PR #912:
URL: https://github.com/apache/lucene/pull/912#issuecomment-1324918186

   Hi, @uschindler!
   
   I'm experimenting with various improvements of I/O in Lucene.
   
   For a base-line, I use the old implementation of MMapDirectory.
   
   The first idea, when I see this code, is to speed up it with a good old Unsafe.
   I did it and got 7-15% in my final queries (we have a proprietary implementation of a search engine on top of Apache Lucene). Comparing to ByteBuffer implementation, it's clear, why the unsafe implementation is faster (don't have range checks, etc).
   
   The second difference, probably you don't need to have a multiple buffers because you can mmap the entire file at once (on 64 bit systems).
   
   Then, I saw the new code on top of a new foreign memory API. I use JDK 17 and the API is a little changed since.
    So, I reimplemented the directory. But I was surprised by the performance regression I got.
   
   My implementation is 1.5-2 slower than ByteBuffer/Unsafe implementation.
   
   May be the problem in performance regressions in JDK 17 (might be fixed in JDK 19 already), or incorrect usage of API (varhandlers are very cranky in terms of C2 opt/inlining).
   
   Before digging into the problem, I'd like to know, do you have any isolated benchmarks of your new implementation and if yes, how it could be compared to the previous implementation?
   
   How readByte() method looks like in my impl.
   ```java
   @Override
   public byte readByte() throws IOException {
     var value = MemoryAccess.getByteAtOffset(memorySegment, position);
     position += Byte.BYTES;
     return value;
   }
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org