[GitHub] [incubator-pinot] sunithabeeram commented on a change in pull request #4270: skip the inverted index generation

[GitBox <gi...@apache.org>]

sunithabeeram commented on a change in pull request #4270: skip the inverted index generation
URL: https://github.com/apache/incubator-pinot/pull/4270#discussion_r289875293
 
 

 ##########
 File path: pinot-core/src/main/java/org/apache/pinot/core/segment/creator/impl/SegmentColumnarIndexCreator.java
 ##########
 @@ -158,8 +158,12 @@ public void init(SegmentGeneratorConfig segmentCreationSpec, SegmentIndexCreatio
         }
 
         // Initialize inverted index creator
+        // Do not created inverted index if sorted, clean up invertedIndexColumns
         if (invertedIndexColumns.contains(columnName)) {
-          if (segmentCreationSpec.isOnHeap()) {
+          if (indexCreationInfo.isSorted()){
+            invertedIndexColumns.remove(columnName);
+            //config.getInvertedIndexCreationColumns().remove(columnName); Do we clean up config?
 
 Review comment:
   The way the execution code is currently structured, for us to use SortedInvertedIndexBasedFilterOperator, we still need the hint for inverted-index hint - so I don't think you can remove it from config. Please trace the code and ensure that that's not broken. Also make sure there are tests covering the execution for sorted columns.
   
   BaseSingleValueQueriesTest.java is a comprehensive test suite that should cover this, but it looks like we are not covering the case there where a column is both sorted and has invertedIndex. You can extend that test to cover this case and ensure things are working as expected. You will most likely also need a test to assert that the bitmap based index isn't generated.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org