You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Sergey Shelukhin (JIRA)" <ji...@apache.org> on 2013/10/31 03:44:25 UTC
[jira] [Created] (HIVE-5705) TopN might use better heuristic for
disable
Sergey Shelukhin created HIVE-5705:
--------------------------------------
Summary: TopN might use better heuristic for disable
Key: HIVE-5705
URL: https://issues.apache.org/jira/browse/HIVE-5705
Project: Hive
Issue Type: Improvement
Reporter: Sergey Shelukhin
Priority: Minor
Right now, if TopN overruns memory threshold it disables itself if it couldn't directly exclude rows as they are sent; it doesn't count evictions that were initially put in the heap and then superceded for this purpose.
It's reasonable in most cases, but if N is relatively small, and map output is large, the cost could still be worth it even if rows don't get excluded. So we'd pay some memory copies but emit much less rows.
--
This message was sent by Atlassian JIRA
(v6.1#6144)