You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Kunal Khatua (JIRA)" <ji...@apache.org> on 2016/06/16 22:58:05 UTC
[jira] [Commented] (DRILL-4236) ExternalSort should use the new
allocator functionality to better manage it's memory usage
[ https://issues.apache.org/jira/browse/DRILL-4236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334890#comment-15334890 ]
Kunal Khatua commented on DRILL-4236:
-------------------------------------
Setting direct memory to 10GB and altered the session by setting `planner.memory.max_query_memory_per_node`=21474836 (20MB).
The data begins to spill to disk with no significant increase in the Drillbit memory footprint.
Verified on Drill 1.6.0.
> ExternalSort should use the new allocator functionality to better manage it's memory usage
> ------------------------------------------------------------------------------------------
>
> Key: DRILL-4236
> URL: https://issues.apache.org/jira/browse/DRILL-4236
> Project: Apache Drill
> Issue Type: Improvement
> Components: Execution - Relational Operators
> Affects Versions: 1.5.0
> Reporter: Deneche A. Hakim
> Assignee: Deneche A. Hakim
> Labels: documentation
> Fix For: 1.5.0
>
>
> Before DRILL-4215, the sort operator wasn't able to correctly compute it's memory usage, and so it jumped through a bunch of hoops to try to figure out when it should spill to disk.
> With the transfer accounting in place, this code can be greatly simplified to just use the current operator memory allocation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)