You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Alex Rodoni (JIRA)" <ji...@apache.org> on 2018/11/08 21:25:00 UTC
[jira] [Work started] (IMPALA-7836) Impala 3.1 Doc: New query
option 'topn_bytes_limit' for TopN to Sort conversion
[ https://issues.apache.org/jira/browse/IMPALA-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on IMPALA-7836 started by Alex Rodoni.
-------------------------------------------
> Impala 3.1 Doc: New query option 'topn_bytes_limit' for TopN to Sort conversion
> -------------------------------------------------------------------------------
>
> Key: IMPALA-7836
> URL: https://issues.apache.org/jira/browse/IMPALA-7836
> Project: IMPALA
> Issue Type: Sub-task
> Components: Docs, Frontend
> Affects Versions: Impala 2.9.0
> Reporter: Sahil Takiar
> Assignee: Alex Rodoni
> Priority: Major
> Labels: future_release_doc
>
> IMPALA-5004 adds a new query level option called 'topn_bytes_limit' that we should document. The changes in IMPALA-5004 work by estimating the amount of memory required to run a TopN operator. The memory estimate is based on the size of the individual tuples that need to be processed by the TopN operator, as well as the sum of the limit and offset in the query. TopN operators don't spill to disk so they have to keep all rows they process in memory.
> If the estimated size of the working set of the TopN operator exceeds the threshold of 'topn_bytes_limit' the TopN operator will be replaced with a Sort operator. The Sort operator can spill to disk, but it processes all the data (the limit and offset have no affect). So switching to Sort might incur performance penalties, but it will require less memory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org