[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #6455: Cleanup dictionary and forward index loading in SegmentPreProcessor

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on a change in pull request #6455:
URL: https://github.com/apache/incubator-pinot/pull/6455#discussion_r560496556



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/segment/index/loader/LoaderUtils.java
##########
@@ -40,6 +50,43 @@ private LoaderUtils() {
 
   private static final Logger LOGGER = LoggerFactory.getLogger(LoaderUtils.class);
 
+  /**
+   * Returns the forward index reader for the given column.
+   */
+  public static ForwardIndexReader<?> getForwardIndexReader(SegmentDirectory.Reader segmentReader,
+      ColumnMetadata columnMetadata)
+      throws IOException {
+    PinotDataBuffer dataBuffer =
+        segmentReader.getIndexFor(columnMetadata.getColumnName(), ColumnIndexType.FORWARD_INDEX);
+    if (columnMetadata.hasDictionary()) {
+      if (columnMetadata.isSingleValue()) {
+        if (columnMetadata.isSorted()) {
+          return new SortedIndexReaderImpl(dataBuffer, columnMetadata.getCardinality());
+        } else {
+          return new FixedBitSVForwardIndexReaderV2(dataBuffer, columnMetadata.getTotalDocs(),
+              columnMetadata.getBitsPerElement());
+        }
+      } else {
+        return new FixedBitMVForwardIndexReader(dataBuffer, columnMetadata.getTotalDocs(),
+            columnMetadata.getTotalNumberOfEntries(), columnMetadata.getBitsPerElement());
+      }
+    } else {
+      DataType dataType = columnMetadata.getDataType();
+      return dataType.isFixedWidth() ? new FixedByteChunkSVForwardIndexReader(dataBuffer, dataType)
+          : new VarByteChunkSVForwardIndexReader(dataBuffer, dataType);
+    }
+  }
+
+  /**
+   * Returns the dictionary reader for the given column.
+   */
+  public static BaseImmutableDictionary getDictionaryReader(SegmentDirectory.Reader segmentReader,

Review comment:
       Updated




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org