[GitHub] [pinot] vvivekiyer commented on a diff in pull request #9885: Fix issues for realtime table reload

[GitBox <gi...@apache.org>]

vvivekiyer commented on code in PR #9885:
URL: https://github.com/apache/pinot/pull/9885#discussion_r1042777559


##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java:
##########
@@ -538,10 +538,14 @@ public void replaceHLSegment(SegmentZKMetadata segmentZKMetadata, IndexLoadingCo
    * Replaces a committed LLC REALTIME segment.
    */
   public void replaceLLSegment(String segmentName, IndexLoadingConfig indexLoadingConfig) {
+    File indexDir = new File(_indexDir, segmentName);
+    // Use the latest table config and schema to load the segment
+    TableConfig tableConfig = ZKMetadataProvider.getTableConfig(_propertyStore, _tableNameWithType);
+    Preconditions.checkState(tableConfig != null, "Failed to get table config for table: {}", _tableNameWithType);
+    Schema schema = ZKMetadataProvider.getTableSchema(_propertyStore, tableConfig);
+    indexLoadingConfig.updateTableConfigAndSchema(tableConfig, schema);

Review Comment:
   If we check the implementation of `updateTableConfigAndSchema()`, we see that only additional indexing configs are added. For example:
   1. If the user removes the inverted index on columnA and updates table config, the change would never be reflected here. That's because we only add new columns where the index is enabled. 
   2. The same is true for others like dictionary, etc.
   
   I can propose a change to use the tableConfig as source of truth but just want to confirm that my understanding is right @Jackie-Jiang 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org