[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #11046: [Bugfix] WIP -- Use PinotDataBuffer For ImmutableFST

["Jackie-Jiang (via GitHub)" <gi...@apache.org>]

Jackie-Jiang commented on code in PR #11046:
URL: https://github.com/apache/pinot/pull/11046#discussion_r1259342992


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/utils/nativefst/NativeFSTIndexReader.java:
##########
@@ -48,8 +51,19 @@ public class NativeFSTIndexReader implements TextIndexReader {
   public NativeFSTIndexReader(PinotDataBuffer dataBuffer)
       throws IOException {
     // TODO: Implement an InputStream directly on PinotDataBuffer
-    ByteBuffer byteBuffer = dataBuffer.toDirectByteBuffer(0, (int) dataBuffer.size());
-    _fst = FST.read(new ByteBufferInputStream(Collections.singletonList(byteBuffer)), ImmutableFST.class, true);
+    /*ByteBuffer byteBuffer = dataBuffer.toDirectByteBuffer(0, (int) dataBuffer.size());

Review Comment:
   We already have `PinotDataBuffer` here, so we should not create an input stream from it, but directly using it to read data. For immutable index, reading from stream is very inefficient because it requires us to copy the data



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org