You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@drill.apache.org by "Pritesh Maker (JIRA)" <ji...@apache.org> on 2018/06/27 17:43:00 UTC
[jira] [Commented] (DRILL-6537) Limit the batch size for buffering
operators based on how much memory they get
[ https://issues.apache.org/jira/browse/DRILL-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16525348#comment-16525348 ]
Pritesh Maker commented on DRILL-6537:
--------------------------------------
[https://github.com/apache/drill/pull/1342] ([~ppenumarthy])
> Limit the batch size for buffering operators based on how much memory they get
> ------------------------------------------------------------------------------
>
> Key: DRILL-6537
> URL: https://issues.apache.org/jira/browse/DRILL-6537
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
> Affects Versions: 1.13.0
> Reporter: Padma Penumarthy
> Assignee: Padma Penumarthy
> Priority: Major
> Fix For: 1.14.0
>
>
> we are using 16MB for output batch size for all operators by default. however, for buffering operators, depending upon how much memory they get allocated, it is possible that 16MB might be too much to use for output batch size, causing them to spill sometimes.
> Have output batch size to be minimum of 16 MB or 20% of the allocated memory.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)