[GitHub] [pinot] Jackie-Jiang commented on pull request #10602: Add a pre-shutdown method to make deleting tables faster

["Jackie-Jiang (via GitHub)" <gi...@apache.org>]

Jackie-Jiang commented on PR #10602:
URL: https://github.com/apache/pinot/pull/10602#issuecomment-1522518227

   > > Can you check if we can directly call shut down? Calling shut down will prevent consuming segment from committing, which is desired
   > 
   > It works when I tried it but then we won't be waiting for EV to converge. It would lead to race condition issues mentioned in #8423
   
   I don't think we will run into the same issue. In #8423, the race condition is triggered when table data manager is shut down without user deleting the table, and the new uploaded segment data gets deleted. The fix for that is to introduce the table delete message which ensures the table is deleted.
   Since we know the table is guaranteed to be deleted, we should be able to just shut down the table data manager, wait for all segments to be cleaned up (EV disappear, or current state disappear which is even better), then remove it from the instance data manager. That is actually similar to the server shut down where we first shut down the data manager, then wait for segments to be OFFLINE


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org