[GitHub] [incubator-druid] leventov opened a new issue #8356: Query execution interleaving

[GitBox <gi...@apache.org>]

leventov opened a new issue #8356: Query execution interleaving
URL: https://github.com/apache/incubator-druid/issues/8356
 
 
   The idea of query execution interleaving first popped up in Druid discussions in the context of parallel result combining in Brokers in the discussion between me and @jihoonson, see https://github.com/apache/incubator-druid/pull/6629#discussion_r241089247.
   
   In general, this is not a new idea to reduce head-of-line blocking effects. See [The Tail at Scale](https://blog.acolyer.org/2015/01/15/the-tail-at-scale/), Dean and Barroso, 2013.
   
   In Druid, interleaving may be implemented inside `PrioritizedExecutorService`.  On Historicals, the element of processing is naturally a segment, when a Broker sends a request to a Historical to process many segments for a large query. On Brokers, interleaving may be implemented as discussed in https://github.com/apache/incubator-druid/pull/6629#discussion_r241089247.
   
   Interleaving will play especially well together with online aggregation (https://github.com/apache/incubator-druid/issues/7087) but would be useful even if implemented on its own independently of online aggregation.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org