You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@tez.apache.org by "Rajesh Balamohan (JIRA)" <ji...@apache.org> on 2016/05/10 13:18:13 UTC

[jira] [Updated] (TEZ-3248) Allowing one fetcher to go past thresholds could cause OOM in corner cases (in OrderedShuffle)

     [ https://issues.apache.org/jira/browse/TEZ-3248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rajesh Balamohan updated TEZ-3248:
----------------------------------
    Attachment: TEZ-3248.1.patch

attaching .1 patch

> Allowing one fetcher to go past thresholds could cause OOM in corner cases (in OrderedShuffle)
> ----------------------------------------------------------------------------------------------
>
>                 Key: TEZ-3248
>                 URL: https://issues.apache.org/jira/browse/TEZ-3248
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Rajesh Balamohan
>            Assignee: Rajesh Balamohan
>         Attachments: TEZ-3248.1.patch
>
>
> In ordered shuffle, tez allows exactly one fetcher to go beyond the thresholds before stalling the fetchers. This works in most of the cases, but in some corner cases could lead to OOM. It might be good to trigger mem to disk merge before stalling the threads to release memory.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)