You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2019/05/02 18:08:00 UTC
[jira] [Created] (IMPALA-8483) Make coordinator fragment
lighter-weight
Tim Armstrong created IMPALA-8483:
-------------------------------------
Summary: Make coordinator fragment lighter-weight
Key: IMPALA-8483
URL: https://issues.apache.org/jira/browse/IMPALA-8483
Project: IMPALA
Issue Type: Improvement
Components: Frontend
Reporter: Tim Armstrong
Assignee: Bikramjeet Vig
it's possible to write queries with an arbitrarily expensive coordinator fragment. One example is a query with a set of unpartitioned analytic functions each with a different ORDER BY, which results in many SORT nodes in the coordinator fragment.
This is a problem for dedicated coordinators because it makes the resource consumption unpredictable.
It would be useful if we could offload that work to executors and guarantee that only "lightweight' operators are part of the coordinator fragment, e.g. operators that take O(#rows returned * log(#executors)) time and O(#executors) memory - regular exchanges, merging exchanges, TOP-N, non-grouping aggregation, etc.
I think this can be done in general by inserting an additional exchange on top of the "expensive" part of a coordinator fragment, and then ensuring that that fragment gets scheduled on an executor.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org