[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8601: Bloom filter during segment creation (WIP)

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on code in PR #8601:
URL: https://github.com/apache/pinot/pull/8601#discussion_r861399927


##########
pinot-segment-local/src/main/java/org/apache/pinot/segment/local/segment/creator/impl/SegmentColumnarIndexCreator.java:
##########
@@ -212,6 +221,19 @@ public void init(SegmentGeneratorConfig segmentCreationSpec, SegmentIndexCreatio
               dictionaryCreator.getNumBytesPerEntry());
           throw e;
         }
+
+        // A Bloom Filter can only be applied to dictionary-encoded columns TODO(saurabh) check on this

Review Comment:
   For settings that cannot be applied to certain encoding, we should inform the users (maybe fail the validation). We may do it in a separate PR.



##########
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/IndexingConfig.java:
##########
@@ -64,6 +64,7 @@ public class IndexingConfig extends BaseJsonConfig {
   // TODO: Add a new configuration related to the segment generation
   private boolean _autoGeneratedInvertedIndex;
   private boolean _createInvertedIndexDuringSegmentGeneration;
+  private boolean _createBloomFilterDuringSegmentGeneration;

Review Comment:
   Don't add this. We can always generate bloom filter during segment generation for now. Adding this can make future change harder because we might not want to have per-index flag



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org