[GitHub] [lucene] dungba88 commented on issue #10520: Improve FST performance created by Builder [LUCENE-9481]

["dungba88 (via GitHub)" <gi...@apache.org>]

dungba88 commented on issue #10520:
URL: https://github.com/apache/lucene/issues/10520#issuecomment-1738333974

   I originally also think of allocating single `byte[]` and yeah it would have the 2X heap size requirement.
   
   As our use case are building and using FST at runtime, there might be (some, I'm not sure) penalty we would have if we are to read/write entirely with filesystem. In some of my previous experience with Java I/O, using FileChannel would be much faster than InputStream/OutputStream when writing to filesystem, but even so they are still slower than main memory.
   
   I'm curious if Lucene also has a benchmark for comparison of off-heap and heap mode? (Also by off-heap, I assume you mean filesystem, instead of the direct memory, e.g with `ByteBuffer.allocateDirect()`?)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org