You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by at...@apache.org on 2023/06/05 06:03:12 UTC
[pinot] branch native_mmap created (now 67aaed4134)
This is an automated email from the ASF dual-hosted git repository.
atri pushed a change to branch native_mmap
in repository https://gitbox.apache.org/repos/asf/pinot.git
at 67aaed4134 Support Off Heap for Native Text Indices
This branch includes the following new commits:
new 67aaed4134 Support Off Heap for Native Text Indices
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[pinot] 01/01: Support Off Heap for Native Text Indices
Posted by at...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
atri pushed a commit to branch native_mmap
in repository https://gitbox.apache.org/repos/asf/pinot.git
commit 67aaed4134e549eb2d739de814bbb503e317170b
Author: Atri Sharma <at...@gmail.com>
AuthorDate: Mon Jun 5 11:32:57 2023 +0530
Support Off Heap for Native Text Indices
This commit allows native text indices to be loaded off heap.
---
.../local/segment/index/readers/text/NativeTextIndexReader.java | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/text/NativeTextIndexReader.java b/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/text/NativeTextIndexReader.java
index 8f1c2e6a73..1dd1a8102d 100644
--- a/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/text/NativeTextIndexReader.java
+++ b/pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/index/readers/text/NativeTextIndexReader.java
@@ -52,7 +52,10 @@ public class NativeTextIndexReader implements TextIndexReader {
public NativeTextIndexReader(String column, File indexDir) {
_column = column;
try {
- _buffer = PinotDataBuffer.loadBigEndianFile(getTextIndexFile(indexDir));
+ String desc = "Native text index buffer: " + column;
+ File indexFile = getTextIndexFile(indexDir);
+ _buffer =
+ PinotDataBuffer.mapFile(indexFile, /* readOnly */ true, 0, indexFile.length(), ByteOrder.BIG_ENDIAN, desc);
populateIndexes();
} catch (Exception e) {
LOGGER.error("Failed to instantiate Lucene text index reader for column {}, exception {}", column,
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org