You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Harsh J (Commented) (JIRA)" <ji...@apache.org> on 2012/04/10 14:29:18 UTC
[jira] [Commented] (MAPREDUCE-4130) Jobid creation is not required
if the job failed because of unavailability of input path.Can input and
output paths validation can be done before job ID creation step?
[ https://issues.apache.org/jira/browse/MAPREDUCE-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250600#comment-13250600 ]
Harsh J commented on MAPREDUCE-4130:
------------------------------------
Also related is MAPREDUCE-2384 (test at least can be reused), and perhaps MAPREDUCE-1588 if it still applies to limits available in Yarn's mrapp.
The splits are currently computed and written in one call, and hence they are to be done after a staging directory is available (with a job ID on its path). I've not seen too many failures in practice at the input split computation level, but the idea of separating these two calls seems alright to me.
> Jobid creation is not required if the job failed because of unavailability of input path.Can input and output paths validation can be done before job ID creation step?
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4130
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4130
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Reporter: Ramgopal N
> Priority: Minor
>
> If the input splits cannot be computed because the input paths doesnt exist
> Job is not submitted and error is thrown.But before that jobid is created which can be avoided.
> The sequence is
> 1)Output path validation
> 2)Jobid creation
> 3)Input splits computation
> But if the sequence of steps is 1,3,2 ...unnecessary jobid creation can be avoided
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira