[GitHub] [incubator-pinot] Jackie-Jiang commented on issue #6222: Uniform the time value in SegmentZKMetadata and support time pruning on broker

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on issue #6222:
URL: https://github.com/apache/incubator-pinot/issues/6222#issuecomment-722027509


   Because of the inconsistent behavior of the LLC table and simple date format described above, we should not rely on the time unit stored in the `SegmentZKMetadata`. To minimize the change, we can keep the current format of the `SegmentZKMetadata`, add APIs to read the start/end time in millis, and deprecate the old getters. We can recover the actual start/end time value with the start/end time in millis and the time format stored in the schema.
   
   - For retention management, use the start/end time in millis.
   - For time boundary management, calculate the time boundary using boundary time in millis and the time format.
   - For time-based segment pruning, calculate the time range in millis from the time range value and the time format, and then prune segments based on the time in millis.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org