[jira] [Created] (DRILL-3896) Allow for dynamic allocation of resources during execution of concurrent queries

["Kunal Khatua (JIRA)" <ji...@apache.org>]

Kunal Khatua created DRILL-3896:
-----------------------------------

             Summary: Allow for dynamic allocation of resources during execution of concurrent queries
                 Key: DRILL-3896
                 URL: https://issues.apache.org/jira/browse/DRILL-3896
             Project: Apache Drill
          Issue Type: Improvement
          Components: Query Planning & Optimization
            Reporter: Kunal Khatua


With increasing number of concurrent queries, we often have to tune the Drill session settings like `planner.width.max_per_node` to reduce committing of resources. Most RDBMS systems dynamically allocate resources as more concurrent queries are running on the system.

A proposal would be to have a flag that enable/disables this dynamic allocation, along with a min/max of the resource. For e.g., on a 10-node setup, the first 4 queries can concurrently run with `planner.width.max_per_node`=23 (default for a 32 core/node system), and every additional 4 queries can reduce this setting for those session by about 2.. down to a minimum value of 6.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)