[GitHub] [incubator-pinot] yupeng9 commented on issue #5627: Query-level priority and control

[GitBox <gi...@apache.org>]

yupeng9 commented on issue #5627:
URL: https://github.com/apache/incubator-pinot/issues/5627#issuecomment-653845032


   > At the minimum, we should leverage the per table level query timeout config (part of table config). Jackie had added this. Server can use this to kill long running queries or the queries that have waited long enough in the scheduler queue and thus are likely to overrun their time limit. Through query options, we can also pass per query time out.
   ah, that's a nice feature. @Jackie-Jiang can we add this config to the docs?
   > 
   > In general, and a potential long term solution is to implement something called as workload management in Pinot to overall improve the multi-tenancy and isolation. I feel we should also revisit the priority token bucket scheduler since FCFS isn't suitable for multitenant deployments.
   > 
   > We can also consider forking a JVM (running in a separate process) so that an extremely bad query (high cpu high gc with the potential to cause OOM) doesn't crash the server.
   I like this idea. Server has two important functionalities of ingestion and querying. I think it is worth consideration of spawning child processes for query execution so that the ingestion is not impacted. 
   
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org