You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@tez.apache.org by "Rajesh Balamohan (JIRA)" <ji...@apache.org> on 2016/05/10 11:36:12 UTC
[jira] [Created] (TEZ-3248) Allowing one fetcher to go past
thresholds could cause OOM in corner cases (in OrderedShuffle)
Rajesh Balamohan created TEZ-3248:
-------------------------------------
Summary: Allowing one fetcher to go past thresholds could cause OOM in corner cases (in OrderedShuffle)
Key: TEZ-3248
URL: https://issues.apache.org/jira/browse/TEZ-3248
Project: Apache Tez
Issue Type: Improvement
Reporter: Rajesh Balamohan
Assignee: Rajesh Balamohan
In ordered shuffle, tez allows exactly one fetcher to go beyond the thresholds before stalling the fetchers. This works in most of the cases, but in some corner cases could lead to OOM. It might be good to trigger mem to disk merge before stalling the threads to release memory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)