[GitHub] [incubator-pinot] jenniferdai commented on a change in pull request #4244: Add support for custom record-readers in the create-segment tool

[GitBox <gi...@apache.org>]

jenniferdai commented on a change in pull request #4244: Add support for custom record-readers in the create-segment tool
URL: https://github.com/apache/incubator-pinot/pull/4244#discussion_r289531085
 
 

 ##########
 File path: pinot-tools/src/main/java/org/apache/pinot/tools/admin/command/CreateSegmentCommand.java
 ##########
 @@ -228,15 +228,17 @@ public boolean execute()
       }
     }
 
-    FileFormat configFormat = segmentGeneratorConfig.getFormat();
-    if (_format == null) {
-      if (configFormat == null) {
-        throw new RuntimeException("Format cannot be null in config file.");
-      }
-      _format = configFormat;
-    } else {
-      if (configFormat != _format && configFormat != FileFormat.AVRO) {
-        LOGGER.warn("Find format conflict in command line and config file, use config in command line: {}", _format);
+    if (segmentGeneratorConfig.getRecordReaderPath() == null) {
 
 Review comment:
       // Allow for instantiation general record readers from a record reader path passed into segment generator config
       // If this is set, this will override the file format
       if (recordReaderPath != null) {
         if (fileFormat != FileFormat.OTHER) {
           // NOTE: we currently have default file format set to AVRO inside segment generator config, do not want to break
           // this behavior for clients.
           LOGGER
               .warn("Using class: {} to read segment, ignoring configured file format: {}", recordReaderPath, fileFormat);
         }
   
   
   Currently, in record reader factory, we have this default. Up to you, but potentially, we could follow the same model though requiring users to set file format to other seems annoying... maybe we can just leave a comment that the behavior is different or put it in relevant docs.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org