[GitHub] [incubator-pinot] yupeng9 commented on issue #5747: Check overlapping offline segments

[GitBox <gi...@apache.org>]

yupeng9 commented on issue #5747:
URL: https://github.com/apache/incubator-pinot/issues/5747#issuecomment-663662162


   Right. The generated segment name includes three parts: time column's min value, time columns' max value, sequence id. e.g. `transcript_OFFLINE_1570863600000_1572418800000_0`. 
   
   We have seen some batch ingestion jobs incorrectly generate the records overlapping on the time column. This reflects on the segment metadata, specifically the min/max range of the segments. It will be helpful to have such inbuilt sanity checks to catch this kind of errors.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org