[jira] Updated: (PIG-781) Error reporting for failed MR jobs

["Gunther Hagleitner (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/PIG-781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Gunther Hagleitner updated PIG-781:
-----------------------------------

    Attachment: partial_failure.patch

This fix associates stores with MR jobs. At the end of the execution it will print out which stores have passed and which ones have failed.

Example:

{noformat}
50% complete
100% complete
1 map reduce job(s) failed!
Failed to produce result in: "hdfs://wilbur11.labs.corp.sp1.yahoo.com/user/hagleitn/baz"
Successfully stored result in: "hdfs://wilbur11.labs.corp.sp1.yahoo.com/user/hagleitn/bar"
Successfully stored result in: "hdfs://wilbur11.labs.corp.sp1.yahoo.com/user/hagleitn/foo"
Some jobs have failed!
{noformat}


> Error reporting for failed MR jobs
> ----------------------------------
>
>                 Key: PIG-781
>                 URL: https://issues.apache.org/jira/browse/PIG-781
>             Project: Pig
>          Issue Type: Improvement
>            Reporter: Gunther Hagleitner
>         Attachments: partial_failure.patch
>
>
> If we have multiple MR jobs to run and some of them fail the behavior of the system is to not stop on the first failure but to keep going. That way jobs that do not depend on the failed job might still succeed.
> The question is to how best report this scenario to a user. How do we tell which jobs failed and which didn't?
> One way could be to tie jobs to stores and report which store locations won't have data and which ones do.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.