You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/04/04 22:47:07 UTC
[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #8398: Allow disabling dict generation for High cardinality columns
Jackie-Jiang commented on code in PR #8398:
URL: https://github.com/apache/pinot/pull/8398#discussion_r842191694
##########
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/IndexingConfig.java:
##########
@@ -53,6 +53,8 @@
private SegmentPartitionConfig _segmentPartitionConfig;
private boolean _aggregateMetrics;
private boolean _nullHandlingEnabled;
+ private boolean _optimizeDictionaryEnabled;
Review Comment:
Suggest renaming to `_optimizeDictionaryForMetrics`
##########
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/IndexingConfig.java:
##########
@@ -53,6 +53,8 @@
private SegmentPartitionConfig _segmentPartitionConfig;
private boolean _aggregateMetrics;
private boolean _nullHandlingEnabled;
+ private boolean _optimizeDictionaryEnabled;
Review Comment:
Let's add some comments here explaining the configs because these 2 configs are not very self-explained
##########
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/IndexingConfig.java:
##########
@@ -53,6 +53,8 @@
private SegmentPartitionConfig _segmentPartitionConfig;
private boolean _aggregateMetrics;
private boolean _nullHandlingEnabled;
+ private boolean _optimizeDictionaryEnabled;
+ private double _thresholdMinPercentDictionaryStorageSaved;
Review Comment:
Suggest changing it to `_dictionarySizeRatioThreshold` (feel free to rename), and make it default to 1.0. In certain cases we might prefer raw index even if its size is slightly larger because that can avoid the two hops lookup, and provide better locality.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org