You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Christian Kunz (JIRA)" <ji...@apache.org> on 2007/04/09 23:01:32 UTC
[jira] Created: (HADOOP-1234) map tasks fail because they do not
find application in file cache
map tasks fail because they do not find application in file cache
-----------------------------------------------------------------
Key: HADOOP-1234
URL: https://issues.apache.org/jira/browse/HADOOP-1234
Project: Hadoop
Issue Type: Bug
Components: mapred
Affects Versions: 0.12.1
Reporter: Christian Kunz
Using C++ map-reduce interface, a lot of tasks fail because they do not find the application in the file cache.
Might be a race condition between multiple tasks on the same node.
java.io.IOException: Cannot run program "<path>/mapred/local/taskTracker/archive/<namenodehost>/<path>/<application>": java.io.IOException: error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
at org.apache.hadoop.mapred.pipes.Application.runClient(Application.java:131)
at org.apache.hadoop.mapred.pipes.Application.(Application.java:59)
at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:44)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:176)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1459)
Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.(UNIXProcess.java:148)
at java.lang.ProcessImpl.start(ProcessImpl.java:65)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1234) map tasks fail because they do not
find application in file cache
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12503940 ]
Hadoop QA commented on HADOOP-1234:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12359394/HADOOP-1234_2_20070611.patch applied and successfully tested against trunk revision r546320.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/270/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/270/console
> map tasks fail because they do not find application in file cache
> -----------------------------------------------------------------
>
> Key: HADOOP-1234
> URL: https://issues.apache.org/jira/browse/HADOOP-1234
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.12.1
> Reporter: Christian Kunz
> Assignee: Arun C Murthy
> Attachments: HADOOP-1234_1_20070514.patch, HADOOP-1234_2_20070611.patch
>
>
> Using C++ map-reduce interface, a lot of tasks fail because they do not find the application in the file cache.
> Might be a race condition between multiple tasks on the same node.
> java.io.IOException: Cannot run program "<path>/mapred/local/taskTracker/archive/<namenodehost>/<path>/<application>": java.io.IOException: error=2, No such file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
> at org.apache.hadoop.mapred.pipes.Application.runClient(Application.java:131)
> at org.apache.hadoop.mapred.pipes.Application.(Application.java:59)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:44)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:176)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1459)
> Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
> ... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1234) map tasks fail because they do not
find application in file cache
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated HADOOP-1234:
---------------------------------
Resolution: Fixed
Fix Version/s: 0.14.0
Status: Resolved (was: Patch Available)
I just committed this. Thanks, Arun!
> map tasks fail because they do not find application in file cache
> -----------------------------------------------------------------
>
> Key: HADOOP-1234
> URL: https://issues.apache.org/jira/browse/HADOOP-1234
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.12.1
> Reporter: Christian Kunz
> Assignee: Arun C Murthy
> Fix For: 0.14.0
>
> Attachments: HADOOP-1234_1_20070514.patch, HADOOP-1234_2_20070611.patch
>
>
> Using C++ map-reduce interface, a lot of tasks fail because they do not find the application in the file cache.
> Might be a race condition between multiple tasks on the same node.
> java.io.IOException: Cannot run program "<path>/mapred/local/taskTracker/archive/<namenodehost>/<path>/<application>": java.io.IOException: error=2, No such file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
> at org.apache.hadoop.mapred.pipes.Application.runClient(Application.java:131)
> at org.apache.hadoop.mapred.pipes.Application.(Application.java:59)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:44)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:176)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1459)
> Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
> ... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1234) map tasks fail because they do not
find application in file cache
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy reassigned HADOOP-1234:
-------------------------------------
Assignee: Arun C Murthy (was: Owen O'Malley)
> map tasks fail because they do not find application in file cache
> -----------------------------------------------------------------
>
> Key: HADOOP-1234
> URL: https://issues.apache.org/jira/browse/HADOOP-1234
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.12.1
> Reporter: Christian Kunz
> Assigned To: Arun C Murthy
> Attachments: HADOOP-1234_1_20070514.patch
>
>
> Using C++ map-reduce interface, a lot of tasks fail because they do not find the application in the file cache.
> Might be a race condition between multiple tasks on the same node.
> java.io.IOException: Cannot run program "<path>/mapred/local/taskTracker/archive/<namenodehost>/<path>/<application>": java.io.IOException: error=2, No such file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
> at org.apache.hadoop.mapred.pipes.Application.runClient(Application.java:131)
> at org.apache.hadoop.mapred.pipes.Application.(Application.java:59)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:44)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:176)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1459)
> Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
> ... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1234) map tasks fail because they do not
find application in file cache
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12505459 ]
Hadoop QA commented on HADOOP-1234:
-----------------------------------
Integrated in Hadoop-Nightly #123 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/123/])
> map tasks fail because they do not find application in file cache
> -----------------------------------------------------------------
>
> Key: HADOOP-1234
> URL: https://issues.apache.org/jira/browse/HADOOP-1234
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.12.1
> Reporter: Christian Kunz
> Assignee: Arun C Murthy
> Fix For: 0.14.0
>
> Attachments: HADOOP-1234_1_20070514.patch, HADOOP-1234_2_20070611.patch
>
>
> Using C++ map-reduce interface, a lot of tasks fail because they do not find the application in the file cache.
> Might be a race condition between multiple tasks on the same node.
> java.io.IOException: Cannot run program "<path>/mapred/local/taskTracker/archive/<namenodehost>/<path>/<application>": java.io.IOException: error=2, No such file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
> at org.apache.hadoop.mapred.pipes.Application.runClient(Application.java:131)
> at org.apache.hadoop.mapred.pipes.Application.(Application.java:59)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:44)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:176)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1459)
> Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
> ... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1234) map tasks fail because they do not
find application in file cache
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-1234:
----------------------------------
Status: Patch Available (was: Open)
> map tasks fail because they do not find application in file cache
> -----------------------------------------------------------------
>
> Key: HADOOP-1234
> URL: https://issues.apache.org/jira/browse/HADOOP-1234
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.12.1
> Reporter: Christian Kunz
> Assignee: Arun C Murthy
> Attachments: HADOOP-1234_1_20070514.patch, HADOOP-1234_2_20070611.patch
>
>
> Using C++ map-reduce interface, a lot of tasks fail because they do not find the application in the file cache.
> Might be a race condition between multiple tasks on the same node.
> java.io.IOException: Cannot run program "<path>/mapred/local/taskTracker/archive/<namenodehost>/<path>/<application>": java.io.IOException: error=2, No such file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
> at org.apache.hadoop.mapred.pipes.Application.runClient(Application.java:131)
> at org.apache.hadoop.mapred.pipes.Application.(Application.java:59)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:44)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:176)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1459)
> Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
> ... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1234) map tasks fail because they do not
find application in file cache
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12490182 ]
Owen O'Malley commented on HADOOP-1234:
---------------------------------------
I think I see the problem. I'll check more on it tomorrow.
> map tasks fail because they do not find application in file cache
> -----------------------------------------------------------------
>
> Key: HADOOP-1234
> URL: https://issues.apache.org/jira/browse/HADOOP-1234
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.12.1
> Reporter: Christian Kunz
> Assigned To: Owen O'Malley
>
> Using C++ map-reduce interface, a lot of tasks fail because they do not find the application in the file cache.
> Might be a race condition between multiple tasks on the same node.
> java.io.IOException: Cannot run program "<path>/mapred/local/taskTracker/archive/<namenodehost>/<path>/<application>": java.io.IOException: error=2, No such file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
> at org.apache.hadoop.mapred.pipes.Application.runClient(Application.java:131)
> at org.apache.hadoop.mapred.pipes.Application.(Application.java:59)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:44)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:176)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1459)
> Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
> ... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1234) map tasks fail because they do not
find application in file cache
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-1234:
----------------------------------
Attachment: HADOOP-1234_2_20070611.patch
Re-submitting patch to reflect changes to trunk...
> map tasks fail because they do not find application in file cache
> -----------------------------------------------------------------
>
> Key: HADOOP-1234
> URL: https://issues.apache.org/jira/browse/HADOOP-1234
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.12.1
> Reporter: Christian Kunz
> Assignee: Arun C Murthy
> Attachments: HADOOP-1234_1_20070514.patch, HADOOP-1234_2_20070611.patch
>
>
> Using C++ map-reduce interface, a lot of tasks fail because they do not find the application in the file cache.
> Might be a race condition between multiple tasks on the same node.
> java.io.IOException: Cannot run program "<path>/mapred/local/taskTracker/archive/<namenodehost>/<path>/<application>": java.io.IOException: error=2, No such file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
> at org.apache.hadoop.mapred.pipes.Application.runClient(Application.java:131)
> at org.apache.hadoop.mapred.pipes.Application.(Application.java:59)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:44)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:176)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1459)
> Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
> ... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HADOOP-1234) map tasks fail because they do not
find application in file cache
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley reassigned HADOOP-1234:
-------------------------------------
Assignee: Owen O'Malley
> map tasks fail because they do not find application in file cache
> -----------------------------------------------------------------
>
> Key: HADOOP-1234
> URL: https://issues.apache.org/jira/browse/HADOOP-1234
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.12.1
> Reporter: Christian Kunz
> Assigned To: Owen O'Malley
>
> Using C++ map-reduce interface, a lot of tasks fail because they do not find the application in the file cache.
> Might be a race condition between multiple tasks on the same node.
> java.io.IOException: Cannot run program "<path>/mapred/local/taskTracker/archive/<namenodehost>/<path>/<application>": java.io.IOException: error=2, No such file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
> at org.apache.hadoop.mapred.pipes.Application.runClient(Application.java:131)
> at org.apache.hadoop.mapred.pipes.Application.(Application.java:59)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:44)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:176)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1459)
> Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
> ... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1234) map tasks fail because they do not
find application in file cache
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HADOOP-1234:
----------------------------------
Attachment: HADOOP-1234_1_20070514.patch
Attached patch for review...
Christian could you see if this works for you? Thanks!
> map tasks fail because they do not find application in file cache
> -----------------------------------------------------------------
>
> Key: HADOOP-1234
> URL: https://issues.apache.org/jira/browse/HADOOP-1234
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.12.1
> Reporter: Christian Kunz
> Assigned To: Arun C Murthy
> Attachments: HADOOP-1234_1_20070514.patch
>
>
> Using C++ map-reduce interface, a lot of tasks fail because they do not find the application in the file cache.
> Might be a race condition between multiple tasks on the same node.
> java.io.IOException: Cannot run program "<path>/mapred/local/taskTracker/archive/<namenodehost>/<path>/<application>": java.io.IOException: error=2, No such file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
> at org.apache.hadoop.mapred.pipes.Application.runClient(Application.java:131)
> at org.apache.hadoop.mapred.pipes.Application.(Application.java:59)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:44)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:176)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1459)
> Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
> ... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1234) map tasks fail because they do not
find application in file cache
Posted by "Devaraj Das (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12489929 ]
Devaraj Das commented on HADOOP-1234:
-------------------------------------
Could you please tell whether you see this problem consistently?
> map tasks fail because they do not find application in file cache
> -----------------------------------------------------------------
>
> Key: HADOOP-1234
> URL: https://issues.apache.org/jira/browse/HADOOP-1234
> Project: Hadoop
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.12.1
> Reporter: Christian Kunz
>
> Using C++ map-reduce interface, a lot of tasks fail because they do not find the application in the file cache.
> Might be a race condition between multiple tasks on the same node.
> java.io.IOException: Cannot run program "<path>/mapred/local/taskTracker/archive/<namenodehost>/<path>/<application>": java.io.IOException: error=2, No such file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
> at org.apache.hadoop.mapred.pipes.Application.runClient(Application.java:131)
> at org.apache.hadoop.mapred.pipes.Application.(Application.java:59)
> at org.apache.hadoop.mapred.pipes.PipesMapRunner.run(PipesMapRunner.java:44)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:176)
> at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1459)
> Caused by: java.io.IOException: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.(UNIXProcess.java:148)
> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
> ... 5 more
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.