[GitHub] [incubator-superset] etr2460 edited a comment on issue #10139: Speed up presto queries!

[GitBox <gi...@apache.org>]

etr2460 edited a comment on issue #10139:
URL: https://github.com/apache/incubator-superset/issues/10139#issuecomment-649068628


   Echoing @villebro and @ktmud, some sort of backoff sounds good, so that a 0.1 second query comes back in 0.2 or 0.4 seconds, but the 30 second query comes back in 32.
   
   I will note that performance around the 1 to 5 second long queries does appear especially bad on Superset's side, especially because of all the steps required to run the query:
   1. receive the request
   2. schedule a task for a worker
   3. pick up the worker task
   4. Run the query (polling ever 2 seconds)
   5. Write the result to the results backend (s3)
   6. Wait for the next poll from the client side (also every 2 seconds I think)
   7. Download the results from s3
   
   This means that a query that takes a second or 2 on cli, can take 2-3 times as long in sql lab. Any improvements we can make around this without overloading services seems worth considering


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org