[GitHub] [incubator-pinot] amrishlal commented on a change in pull request #6922: Enhance PinotSegmentRecordReader to preserve null values

[GitBox <gi...@apache.org>]

amrishlal commented on a change in pull request #6922:
URL: https://github.com/apache/incubator-pinot/pull/6922#discussion_r633860655



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/minion/SegmentConverter.java
##########
@@ -89,6 +95,21 @@ public SegmentConverter(List<File> inputIndexDirs, File workingDir, String table
     _recordAggregator = recordAggregator;
     _groupByColumns = groupByColumns;
     _skipTimeValueCheck = skipTimeValueCheck;
+
+    // Validate that segment schemas from all segments are the same
+    Set<Schema> schemas = new HashSet<>();
+    for (File indexDir : inputIndexDirs) {
+      try {
+        schemas.add(new SegmentMetadataImpl(indexDir).getSchema());
+      } catch (IOException e) {
+        throw new RuntimeException("Caught exception while reading schema from: " + indexDir);
+      }
+    }
+    if (schemas.size() == 1) {
+      _schema = schemas.iterator().next();
+    } else {
+      throw new IllegalStateException("Schemas from input segments are not the same");
+    }

Review comment:
       How about bailing out early from the for loop if a new schema is encountered instead of continuing to process all the files.
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org