You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Runping Qi (JIRA)" <ji...@apache.org> on 2006/09/25 21:51:50 UTC
[jira] Created: (HADOOP-557) task tracker cannot find mapoutput
files
task tracker cannot find mapoutput files
-----------------------------------------
Key: HADOOP-557
URL: http://issues.apache.org/jira/browse/HADOOP-557
Project: Hadoop
Issue Type: Bug
Reporter: Runping Qi
I ran a large job on our cluster over the weekend. At some point, some map tasks were re-run successfully. However, when trying to handle the reducers requests to fetch mapoutput files of those map tasks, the http server complained file not found.
This happened multiple times on different machines for the same map task until the job tracker aborted the job.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-557) task tracker cannot find mapoutput
files
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-557?page=comments#action_12437700 ]
Runping Qi commented on HADOOP-557:
-----------------------------------
The problem was due to config file update during the job running. The job tracker used the original version, but the newly launched tasks used the new one. The new config file specifies a new mapred.local.dir which the task tracker was not aware. When requests came for the mapoutput files, the task tracker generated wrong file names.
Solution suggestions: The child process should inherit config data from the task tracker in order to keep config data consistency.
> task tracker cannot find mapoutput files
> ----------------------------------------
>
> Key: HADOOP-557
> URL: http://issues.apache.org/jira/browse/HADOOP-557
> Project: Hadoop
> Issue Type: Bug
> Reporter: Runping Qi
>
> I ran a large job on our cluster over the weekend. At some point, some map tasks were re-run successfully. However, when trying to handle the reducers requests to fetch mapoutput files of those map tasks, the http server complained file not found.
> This happened multiple times on different machines for the same map task until the job tracker aborted the job.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-557) task tracker cannot find mapoutput
files
Posted by "Runping Qi (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-557?page=comments#action_12437661 ]
Runping Qi commented on HADOOP-557:
-----------------------------------
I ran a large job on our cluster over the weekend. At some point, some map tasks were re-run successfully. However, when trying to handle the reducers requests to fetch mapoutput files of those map tasks, the http server complained file not found. That caused the map tasks to be reported as having been lost, and failed.
This happened multiple times on different machines for the same map task until the job tracker aborted the job.
> task tracker cannot find mapoutput files
> ----------------------------------------
>
> Key: HADOOP-557
> URL: http://issues.apache.org/jira/browse/HADOOP-557
> Project: Hadoop
> Issue Type: Bug
> Reporter: Runping Qi
>
> I ran a large job on our cluster over the weekend. At some point, some map tasks were re-run successfully. However, when trying to handle the reducers requests to fetch mapoutput files of those map tasks, the http server complained file not found.
> This happened multiple times on different machines for the same map task until the job tracker aborted the job.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-557) task tracker cannot find mapoutput
files
Posted by "Sameer Paranjpye (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-557?page=all ]
Sameer Paranjpye updated HADOOP-557:
------------------------------------
Component/s: mapred
Description:
I ran a large job on our cluster over the weekend. At some point, some map tasks were re-run successfully. However, when trying to handle the reducers requests to fetch mapoutput files of those map tasks, the http server complained file not found.
This happened multiple times on different machines for the same map task until the job tracker aborted the job.
was:
I ran a large job on our cluster over the weekend. At some point, some map tasks were re-run successfully. However, when trying to handle the reducers requests to fetch mapoutput files of those map tasks, the http server complained file not found.
This happened multiple times on different machines for the same map task until the job tracker aborted the job.
> task tracker cannot find mapoutput files
> ----------------------------------------
>
> Key: HADOOP-557
> URL: http://issues.apache.org/jira/browse/HADOOP-557
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Reporter: Runping Qi
>
> I ran a large job on our cluster over the weekend. At some point, some map tasks were re-run successfully. However, when trying to handle the reducers requests to fetch mapoutput files of those map tasks, the http server complained file not found.
> This happened multiple times on different machines for the same map task until the job tracker aborted the job.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira