[GitHub] [incubator-pinot] mcvsubbu commented on issue #6679: Admin command to recover from deleted CONSUMING segment case

[GitBox <gi...@apache.org>]

mcvsubbu commented on issue #6679:
URL: https://github.com/apache/incubator-pinot/issues/6679#issuecomment-875911193


   Currently (unless through human intervention), it is never possible that IDEALSTATE does not contain a CONSUMING segment for each partition(group). I therefore agree with Neha, that if the consuming segment is deleted (for whatever reason) by the administrator, the admin should re-create the consuming segment via human interaction.
   
   So, we can create an Admin API that allows the operator to recreate the CONSUMING segment. Also, have an option in there to pick a very high sequence number. This will also help address issues like #7100 
   
   In general, this can be a way to recover realtime tables from extreme scenarios (like changing kafka topics underneath, etc.)
   
   > @npawar @kriti-sc This scenario should be handled within `PinotLLCRealtimeSegmentManager.ensureAllPartitionsConsuming()` when a partition does not have any consuming segment in ideal state. We should pick the end offset of the latest completed segment and put it as the start offset of the new created consuming segment.v


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org