You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues-all@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2020/03/17 15:43:00 UTC

[jira] [Created] (IMPALA-9530) Allow limiting memoy consumed by preaggregation

Tim Armstrong created IMPALA-9530:
-------------------------------------

             Summary: Allow limiting memoy consumed by preaggregation
                 Key: IMPALA-9530
                 URL: https://issues.apache.org/jira/browse/IMPALA-9530
             Project: IMPALA
          Issue Type: Improvement
          Components: Backend
            Reporter: Tim Armstrong
            Assignee: Tim Armstrong


In some cases pre-aggregations can balloon up and consume lots of memory, forcing the merge aggregation to spill. This is often OK as long as the preaggregation is reducing the input sufficiently, since it reduces the amount of data shuffled over the network.  

However in some cases it's preferable to be more conservative with memory and just cap the size of the preaggregation to prevent it ballooning too much.

It would be useful to add a query option to directly limit the memory consumption of the preaggregations.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org