[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #4693: Fetch new schema when loading committed real-time segments

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on a change in pull request #4693: Fetch new schema when loading committed real-time segments
URL: https://github.com/apache/incubator-pinot/pull/4693#discussion_r333283782
 
 

 ##########
 File path: pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java
 ##########
 @@ -205,9 +196,18 @@ public void notifySegmentCommitted(String tableNameWithType, RealtimeSegmentZKMe
   public void addSegment(@Nonnull String segmentName, @Nonnull TableConfig tableConfig,
       @Nonnull IndexLoadingConfig indexLoadingConfig)
       throws Exception {
+    SegmentDataManager segmentDataManager = _segmentDataManagerMap.get(segmentName);
+    if (segmentDataManager != null) {
+      _logger.warn("Skipping adding existing segment: {} for table: {} with data manager class: {}", segmentName,
+          _tableNameWithType, segmentDataManager.getClass().getSimpleName());
+      return;
+    }
+
     RealtimeSegmentZKMetadata realtimeSegmentZKMetadata =
         ZKMetadataProvider.getRealtimeSegmentZKMetadata(_propertyStore, _tableNameWithType, segmentName);
     Preconditions.checkNotNull(realtimeSegmentZKMetadata);
+    Schema schema = ZKMetadataProvider.getTableSchema(_propertyStore, _tableNameWithType);
+    Preconditions.checkNotNull(schema);
 
 Review comment:
   This code path is shared for committing segment as well as new consuming segment (check line242 in old file).
   For real-time table, schema is mandatory, so this should not cause any problem.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org