You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Siddharth Seth (JIRA)" <ji...@apache.org> on 2017/02/02 02:03:52 UTC

[jira] [Commented] (TEZ-3244) Allow overlap of input and output memory when they are not concurrent

    [ https://issues.apache.org/jira/browse/TEZ-3244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849284#comment-15849284 ] 

Siddharth Seth commented on TEZ-3244:
-------------------------------------

bq. We should fix TEZ-3195. But it is still good to have the Input special handling config. For cases like replicated join, were the input would be read and cached in memory the processor will also require more memory. In those cases giving more memory to inputs can cause OOM and so will have to be turned off.
Makes sense. I still haven't figured out the exact issues in TEZ-3195 though.

On the ScalingDistributor - maybe we should just doc these parameters to say they don't work with them.

> Allow overlap of input and output memory when they are not concurrent
> ---------------------------------------------------------------------
>
>                 Key: TEZ-3244
>                 URL: https://issues.apache.org/jira/browse/TEZ-3244
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>         Attachments: TEZ-3244.001.patch, TEZ-3244.002.patch
>
>
> For cases when memory for inputs and outputs are not needed simultaneously it would be more efficient to allow inputs to use the memory normally set aside for outputs and vice-versa.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)