[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #9527: Do not create dictionary for high-cardinality columns

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on code in PR #9527:
URL: https://github.com/apache/pinot/pull/9527#discussion_r993085274


##########
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/IndexingConfig.java:
##########
@@ -55,10 +55,11 @@ public class IndexingConfig extends BaseJsonConfig {
   private boolean _nullHandlingEnabled;
 
   /**
-   * If `optimizeDictionaryForMetrics` enabled, dictionary is not created for the metric columns
-   * for which rawIndexSize / forwardIndexSize is less than the `noDictionarySizeRatioThreshold`.
+   * If `optimizeDictionaryForMetrics` enabled, dictionary is not created for the high-cardinality
+   * single-valued columns for which rawIndexSize / forwardIndexSize is less than the `noDictionarySizeRatioThreshold`.
+   * It also disables dictionary for json and text columns.
    */
-  private boolean _optimizeDictionaryForMetrics;
+  private boolean _optimizeDictionary;

Review Comment:
   Long term wise, we want to have a flag to automatically choose whether to use fixed-length dictionary, var-length dictionary or raw index. Not sure if we want to name it `optimizeDictionary`, but I don't have a good name either



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org