You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Owen O'Malley (JIRA)" <ji...@apache.org> on 2006/02/06 20:46:57 UTC
[jira] Created: (HADOOP-23) single node cluster gets one reducer
single node cluster gets one reducer
------------------------------------
Key: HADOOP-23
URL: http://issues.apache.org/jira/browse/HADOOP-23
Project: Hadoop
Type: Bug
Components: mapred
Reporter: Owen O'Malley
Priority: Minor
Running on a single node cluster (it runs a job tracker and a single task tracker), even though my application asks for 7 reduces, it only gets one. I haven't tracked down what is happening yet.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-23) single node cluster gets one reducer
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-23?page=comments#action_12365359 ]
Owen O'Malley commented on HADOOP-23:
-------------------------------------
Ok, I found the problem and it was a config file ordering problem.
The problem is that the job's config file is loaded as a default resource and the site config file is a final resource and I had set the number of reduces to one in the site file, because on a single node cluster that seemed like a reasonable default.
Obviously, removing the default in the site file is necessary.
I realize now that is why you created the separate mapred-default.xml. It doesn't feel right to have a config file that is specific for map/reduce, but I think there should be some kind of default config file that the user can override in the job but that is editted by the site.
Things that probably belong there:
fs.default.name
mapred.job.tracker
mapred.map.tasks
mapred.reduce.tasks
> single node cluster gets one reducer
> ------------------------------------
>
> Key: HADOOP-23
> URL: http://issues.apache.org/jira/browse/HADOOP-23
> Project: Hadoop
> Type: Bug
> Components: mapred
> Reporter: Owen O'Malley
> Priority: Minor
>
> Running on a single node cluster (it runs a job tracker and a single task tracker), even though my application asks for 7 reduces, it only gets one. I haven't tracked down what is happening yet.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-23) single node cluster gets one reducer
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-23?page=comments#action_12365335 ]
Doug Cutting commented on HADOOP-23:
------------------------------------
Look closely at the order the config files are loaded. This code has been in flux, and may have bugs.
> single node cluster gets one reducer
> ------------------------------------
>
> Key: HADOOP-23
> URL: http://issues.apache.org/jira/browse/HADOOP-23
> Project: Hadoop
> Type: Bug
> Components: mapred
> Reporter: Owen O'Malley
> Priority: Minor
>
> Running on a single node cluster (it runs a job tracker and a single task tracker), even though my application asks for 7 reduces, it only gets one. I haven't tracked down what is happening yet.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (HADOOP-23) single node cluster gets one reducer
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-23?page=all ]
Owen O'Malley resolved HADOOP-23:
---------------------------------
Resolution: Won't Fix
> single node cluster gets one reducer
> ------------------------------------
>
> Key: HADOOP-23
> URL: http://issues.apache.org/jira/browse/HADOOP-23
> Project: Hadoop
> Type: Bug
> Components: mapred
> Reporter: Owen O'Malley
> Priority: Minor
>
> Running on a single node cluster (it runs a job tracker and a single task tracker), even though my application asks for 7 reduces, it only gets one. I haven't tracked down what is happening yet.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira