[GitHub] [pinot] Jackie-Jiang commented on a diff in pull request #9885: Fix issues for realtime table reload

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on code in PR #9885:
URL: https://github.com/apache/pinot/pull/9885#discussion_r1042784136


##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java:
##########
@@ -538,10 +538,14 @@ public void replaceHLSegment(SegmentZKMetadata segmentZKMetadata, IndexLoadingCo
    * Replaces a committed LLC REALTIME segment.
    */
   public void replaceLLSegment(String segmentName, IndexLoadingConfig indexLoadingConfig) {
+    File indexDir = new File(_indexDir, segmentName);
+    // Use the latest table config and schema to load the segment
+    TableConfig tableConfig = ZKMetadataProvider.getTableConfig(_propertyStore, _tableNameWithType);
+    Preconditions.checkState(tableConfig != null, "Failed to get table config for table: {}", _tableNameWithType);
+    Schema schema = ZKMetadataProvider.getTableSchema(_propertyStore, tableConfig);
+    indexLoadingConfig.updateTableConfigAndSchema(tableConfig, schema);

Review Comment:
   Yes, index removal is not properly handled here.
   I think the easy fix is to add `InstanceDataManagerConfig getInstanceDataManagerConfig()` into `IndexLoadingConfig`, and create a new `IndexLoadingConfig` here.
   Currently `IndexLoadingConfig` takes `InstanceDataManagerConfig`, `TableConfig` and `Schema`, then re-create all configs again from them. Ideally we can remove it or make it a thin wrapper on top of these configs, which will prevent such problems, and make it much easier to manage



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org