[jira] [Created] (IMPALA-10025) Avoid rebuilding in-memory heap during output phase of top-n

["Tim Armstrong (Jira)" <ji...@apache.org>]

Tim Armstrong created IMPALA-10025:
--------------------------------------

             Summary: Avoid rebuilding in-memory heap during output phase of top-n
                 Key: IMPALA-10025
                 URL: https://issues.apache.org/jira/browse/IMPALA-10025
             Project: IMPALA
          Issue Type: Sub-task
          Components: Backend
            Reporter: Tim Armstrong
            Assignee: Tim Armstrong


In the patch for IMPALA-9853, we reuse some code in the output phase that necessitated building the in-memory heap from the sorter's output. This has some inherent overhead that gets worse for larger limits and/or partition counts.

It would be better to have the sorter do a full sort on partition/order by columns and then apply the limit while streaming the results back from the sorter. In combination with IMPALA-10023 this would let us gracefully degrade to doing something closer to a regular sort and probably let us bump ANALYTIC_PUSHDOWN_THRESHOLD.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org