[GitHub] [pinot] npawar commented on a diff in pull request #10553: Allow to overwrite index configs at tier level

["npawar (via GitHub)" <gi...@apache.org>]

npawar commented on code in PR #10553:
URL: https://github.com/apache/pinot/pull/10553#discussion_r1169235226


##########
pinot-common/src/main/java/org/apache/pinot/common/utils/config/TableConfigUtils.java:
##########
@@ -331,6 +339,96 @@ public static void convertFromLegacyTableConfig(TableConfig tableConfig) {
     validationConfig.setSegmentPushType(null);
   }
 
+  /**
+   * Helper method to create a new TableConfig by overwriting the original TableConfig with tier specific configs, so
+   * that the consumers of TableConfig don't have to handle tier overwrites themselves. To begin with, we only
+   * consider to overwrite the index configs in `tableIndexConfig` and `fieldConfigList`, e.g.
+   *
+   * {
+   *   "tableIndexConfig": {
+   *     ... // configs allowed in IndexingConfig, for default tier
+   *     "tierOverwrites": {
+   *       "hotTier": {...}, // configs allowed in IndexingConfig, for hot tier
+   *       "coldTier": {...} // configs allowed in IndexingConfig, for cold tier
+   *     }
+   *   }
+   *   "fieldConfigList": [
+   *     {
+   *       ... // configs allowed in FieldConfig, for default tier
+   *       "tierOverwrites": {
+   *         "hotTier": {...}, // configs allowed in FieldConfig, for hot tier
+   *         "coldTier": {...} // configs allowed in FieldConfig, for cold tier
+   *       }
+   *     },
+   *     ...
+   *   ]
+   * }

Review Comment:
   how does this work given tableIndexConfig is at table level, vs fieldConfigList is at column level? What exactly is one overriding when adding in tableIndexing? Or is the instruction to just repeat whatever json fields one is interested in ?



##########
pinot-spi/src/main/java/org/apache/pinot/spi/utils/builder/TableConfigBuilder.java:
##########
@@ -460,6 +467,7 @@ public TableConfig build() {
     indexingConfig.setAggregateMetrics(_aggregateMetrics);
     indexingConfig.setOptimizeDictionaryForMetrics(_optimizeDictionaryForMetrics);
     indexingConfig.setNoDictionarySizeRatioThreshold(_noDictionarySizeRatioThreshold);
+    indexingConfig.setTierOverwrites(_tierOverwrites);

Review Comment:
   why is this only being set in indexingConfig and not in fieldConfigList? or rather - how does one know which overwrites on is providing via the setter?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org