Re: [I] Pauseless Consumption [pinot]

["kishoreg (via GitHub)" <gi...@apache.org>]

kishoreg commented on issue #10147:
URL: https://github.com/apache/pinot/issues/10147#issuecomment-2182011009

   Reviving this. 
   
   I think we should go with optimistic approach as Jackie suggested. One simplification would be to enable pauseless consumption feature only in case of commit happening because of reaching row threshold. This will always ensure that the ending offset is the same on all replicas. There is no harm in going ahead with next segment immediately in this case.
   
   The edge case happens when the commit is decided based on time because each replica can commit at different offset. To avoid all edge cases, we can simply fallback to current behavior in this case.
   
   Thoughts?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org