[GitHub] [pinot] snleee opened a new issue #7813: Supporting Consistent Data Push and Data Rollback on Pinot

[GitBox <gi...@apache.org>]

snleee opened a new issue #7813:
URL: https://github.com/apache/pinot/issues/7813


   @jackjlli and I have come up with the design document for supporting the consistent push on Pinot. The main idea is to reuse the "segment replacement protocol" added as part of the merge-rollup service to switch old segments with new segments.
   
   The existing protocol has some limitations since it was originally designed in the context of merge/rollup. Here are some main changes:
   
   - Add "reverted" state for segment lineage entry & revert API
   - Add "select" API that returns the list of target segments to be replaced given the time range.
   - Improve the segment name generator for the unique segment name
   - Improve the code to cover some failure cases
   
   https://docs.google.com/document/d/1PUy4wSUPFyEWEW3a88Mipdug3cPj4EpV__lx-BVUTYk/edit#


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org