[jira] [Created] (DRILL-2698) Add allocation policy for (direct memory) limits per-query-per-node

["Chris Westin (JIRA)" <ji...@apache.org>]

Chris Westin created DRILL-2698:
-----------------------------------

             Summary: Add allocation policy for (direct memory) limits per-query-per-node
                 Key: DRILL-2698
                 URL: https://issues.apache.org/jira/browse/DRILL-2698
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 0.8.0
            Reporter: Chris Westin
            Assignee: Chris Westin
             Fix For: 1.0.0


We want the allocator to support a policy that limits memory allocation per-query-per-node. Current policy is to divide the available direct memory by the number of fragments running on a node, and limit each fragment to that. This new policy would count fragments that belong to the same query and are running on the same node together. So, if query A has 3 fragments running on a node, and query B has 5 fragments running on the same node, and there are no other queries with fragments on that node, then there are two queries running on the node, and their allocation limit is the available direct memory divided by two. (Under the original policy, which divides the available memory by the number of fragments, A would be allowed 3/8 of the memory, and B would be allowed 5/8 of the memory.)

Note that this policy should be a user-settable boot-time option; the original policy would still also be available as an option.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)