You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2021/03/10 18:12:16 UTC

[GitHub] [incubator-pinot] jackjlli commented on pull request #6664: Add an option to fail segment creation job when getting empty files

jackjlli commented on pull request #6664:
URL: https://github.com/apache/incubator-pinot/pull/6664#issuecomment-795846709


   > I don't think this handling all the cases properly. This is just checking if recordreader.next is empty which is handling empty input file but it's not handling the case where all rows get filtered. It's probably better to check for 0 records at the seal which will catch a lot of other use cases.
   > 
   > Also, this flag is not being passed from the segmentGeneration command
   > 
   > @npawar please take a look at this.
   
   That's good point. While I think it's worthwhile to check empty input file before segment creation so that all the object instantiations can be skipped. Will add one more check in `handlePostCreation` method and add the option in `CreateSegmentCommand`.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org