[GitHub] [incubator-pinot] Jackie-Jiang opened a new pull request #4764: Fix the bug of releasing the segment when there are still threads working on it

[GitBox <gi...@apache.org>]

Jackie-Jiang opened a new pull request #4764: Fix the bug of releasing the segment when there are still threads working on it
URL: https://github.com/apache/incubator-pinot/pull/4764
 
 
   Issue:
   When we execute a query, the main thread holds the segment reference,
   and avoid the segment getting released during the query execution.
   When the query timed out, the main thread will return the exception
   and release the segment reference without waiting for all the worker
   threads finish the work. This can cause severe issue when segment get
   released due to segment deletion/refresh while worker threads are
   still accessing it, even JVM crash.
   
   Solution:
   Use a Phaser for the query execution to ensure when the main thread
   returns the result, all the worker threads are not accessing the
   segment (not scheduled or done). To shorten the period of waiting,
   check the interrupt flag when the operator starts processing the next
   block, and interrupt the execution if the thread is interrupted.
   
   Applied the solution to both planning phase and execution phase, and
   also added tests to check the behavior.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org