You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Devaraj K (JIRA)" <ji...@apache.org> on 2011/02/03 16:58:29 UTC

[jira] Created: (MAPREDUCE-2297) All map reduce tasks are failing if we give invalid path jar file for Job

All map reduce tasks are failing if we give invalid path jar file for Job
-------------------------------------------------------------------------

                 Key: MAPREDUCE-2297
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2297
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: tasktracker
    Affects Versions: 0.20.2
            Reporter: Devaraj K
            Priority: Minor


This can be reproduced by giving the invalid jar file for the Job or it can be reproduced from hive.


In hive-default.xml

<property>
<name>hive.aux.jars.path</name>
<value></value>
<description>Provided for adding auxillaryjarsPath</description>
</property>

If we configure an invalid path for jar file, It is making all map reduce tasks to fail even those jobs are not depending on this jar file and it is giving the below exception.
{code:xml} 
hive> select * from a join b on(a.b=b.c);
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
java.io.FileNotFoundException: File does not exist: /user/root/grade.jar
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:495)
at org.apache.hadoop.filecache.DistributedCache.getTimestamp(DistributedCache.java:509)
at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:651)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:783)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:752)
at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:698)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:64)

{code} 

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (MAPREDUCE-2297) All map reduce tasks are failing if we give invalid path jar file for Job

Posted by "Devaraj K (Resolved) (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/MAPREDUCE-2297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Devaraj K resolved MAPREDUCE-2297.
----------------------------------

    Resolution: Not A Problem

It is no more problem in active versions.
                
> All map reduce tasks are failing if we give invalid path jar file for Job
> -------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2297
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2297
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: tasktracker
>    Affects Versions: 0.20.2
>            Reporter: Devaraj K
>            Assignee: Devaraj K
>            Priority: Minor
>         Attachments: MAPREDUCE-2297.patch
>
>
> This can be reproduced by giving the invalid jar file for the Job or it can be reproduced from hive.
> In hive-default.xml
> <property>
> <name>hive.aux.jars.path</name>
> <value></value>
> <description>Provided for adding auxillaryjarsPath</description>
> </property>
> If we configure an invalid path for jar file, It is making all map reduce tasks to fail even those jobs are not depending on this jar file and it is giving the below exception.
> {code:xml} 
> hive> select * from a join b on(a.b=b.c);
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks not specified. Estimated from input data size: 1
> In order to change the average load for a reducer (in bytes):
> set hive.exec.reducers.bytes.per.reducer=<number>
> In order to limit the maximum number of reducers:
> set hive.exec.reducers.max=<number>
> In order to set a constant number of reducers:
> set mapred.reduce.tasks=<number>
> java.io.FileNotFoundException: File does not exist: /user/root/grade.jar
> at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:495)
> at org.apache.hadoop.filecache.DistributedCache.getTimestamp(DistributedCache.java:509)
> at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:651)
> at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:783)
> at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:752)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:698)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:107)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:64)
> {code} 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira