You are viewing a plain text version of this content. The canonical link for it is here.

Posted to yarn-issues@hadoop.apache.org by "Sunil G (JIRA)" <ji...@apache.org> on 2014/01/28 08:00:44 UTC

[jira] [Created] (YARN-1662) Capacity Scheduler reservation issue cause Job Hang

Sunil G created YARN-1662:
-----------------------------

             Summary: Capacity Scheduler reservation issue cause Job Hang
                 Key: YARN-1662
                 URL: https://issues.apache.org/jira/browse/YARN-1662
             Project: Hadoop YARN
          Issue Type: Bug
          Components: resourcemanager
    Affects Versions: 2.2.0
         Environment: Suse 11 SP1 + Linux
            Reporter: Sunil G


There are 2 node managers in my cluster.
NM1 with 8GB
NM2 with 8GB

I am submitting a Job with below details:
AM with 2GB
Map needs 5GB
Reducer needs 3GB
slowstart is enabled with 0.5
10maps and 50reducers are assigned.

5maps are completed. Now few reducers got scheduled.

Now NM1 has 2GB AM and 3Gb Reducer_1    [Used 5GB]
NM2 has 3Gb Reducer_2			         [Used 3GB]

A Map has now reserved(5GB) in NM1 which has only 3Gb free.
It hangs forever.

Potential issue is, reservation is now blocked in NM1 for a Map which needs 5GB.
But the Reducer_1 hangs by waiting for few map ouputs.

Reducer side preemption also not happened as few headroom is still available.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)