[GitHub] snleee commented on a change in pull request #3895: Fix the RecordReader to pick incoming time column

[GitBox <gi...@apache.org>]

snleee commented on a change in pull request #3895: Fix the RecordReader to pick incoming time column
URL: https://github.com/apache/incubator-pinot/pull/3895#discussion_r261494255
 
 

 ##########
 File path: pinot-core/src/main/java/org/apache/pinot/core/data/readers/AvroRecordReader.java
 ##########
 @@ -60,26 +64,58 @@ public AvroRecordReader(File dataFile, Schema schema)
   private void validateSchema() {
     org.apache.avro.Schema avroSchema = _avroReader.getSchema();
     for (FieldSpec fieldSpec : _schema.getAllFieldSpecs()) {
-      String fieldName = fieldSpec.getName();
-      Field avroField = avroSchema.getField(fieldName);
-      if (avroField == null) {
-        LOGGER.warn("Pinot field: {} does not exist in Avro Schema", fieldName);
-      } else {
-        boolean isPinotFieldSingleValue = fieldSpec.isSingleValueField();
-        boolean isAvroFieldSingleValue = AvroUtils.isSingleValueField(avroField);
-        if (isPinotFieldSingleValue != isAvroFieldSingleValue) {
-          String errorMessage = "Pinot field: " + fieldName + " is " + (isPinotFieldSingleValue ? "Single" : "Multi")
-              + "-valued in Pinot schema but not in Avro schema";
+      if (fieldSpec.getFieldType() == FieldSpec.FieldType.TIME) {
 
 Review comment:
   Is there a way to avoid this repeating validation logic for each record reader? This makes hard for people to add a new type of record reader. For instance, outside contributor is working on adding a parquet reader https://github.com/apache/incubator-pinot/pull/3852

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org