You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by Jim Colestock <jc...@ramblingredneck.com> on 2013/08/27 14:59:59 UTC

reduce job hung in pending state: "No room for reduce task"

Hello All, 

We're running into the following 2 bugs again: 
https://issues.apache.org/jira/browse/HADOOP-5241
https://issues.apache.org/jira/browse/MAPREDUCE-2324

Both of them a listed as closed fixed.  (I was actually the one that got cloudera to submit MAPREDUCE-2324)  Does anyone know is anyone else seeing these in later releases?   We're running the following on various version of Cent OS with Java 1.6:

hadoop-2.0.0+1357-1.cdh4.3.0.p0.21.el5

hadoop-0.20-mapreduce-jobtracker-2.0.0+1357-1.cdh4.3.0.p0.21.el5
hadoop-0.20-mapreduce-2.0.0+1357-1.cdh4.3.0.p0.21.el5
hadoop-0.20-mapreduce-tasktracker-2.0.0+1357-1.cdh4.3.0.p0.21.el5

hadoop-hdfs-namenode-2.0.0+1357-1.cdh4.3.0.p0.21.el5
hadoop-hdfs-secondarynamenode-2.0.0+1357-1.cdh4.3.0.p0.21.el5
hadoop-hdfs-2.0.0+1357-1.cdh4.3.0.p0.21.el5
hadoop-hdfs-datanode-2.0.0+1357-1.cdh4.3.0.p0.21.el5

Just for a quick summary, basically a reduce job get hung in pending while trying to find room on a task tracker, it keeps trying over and over and never fails.  So you end up with a whole bunch of these in the logs: 

2013-08-27 00:48:01,412 WARN org.apache.hadoop.mapred.JobInProgress: No room for reduce task. Node tracker_104.sm.tld:127.0.0.1/127.0.0.1:43723 has 250176954368 bytes free; but we expect reduce input to take 283580756533

Thanks in advance for any help on the issue.. 

JC