[jira] Updated: (HADOOP-1857) Ability to run a script when a task fails to capture stack traces

["arkady borkovsky (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-1857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

arkady borkovsky updated HADOOP-1857:
-------------------------------------


For streaming, it would be good to have some default failure handling.
It would be nice if the default task failure handling covers the following:

First of all: the message in the UI should state that the streaming command has failed

Second: Runping' suggestion about the current record and tasks stats is most useful.
Add to this 
ls -l for the current directory and 
getenv

Third: a few regexp patterns can capture most of the typical failures:
   * shell error message (command not found, wrong permissions, specific messages from awk, grep, sed, etc.)
   * Perl and Python stack traces from the stderr
   * if a core file is present -- print its stack.
This will cover most of the problems.

Also: certain type of errors -- command not find, syntax error in a script, etc -- should kill the job without retries.
If an error of this kind happened in one task (== re pattern matched), it will happen in all tasks. 

> Ability to run a script when a task fails to capture stack traces
> -----------------------------------------------------------------
>
>                 Key: HADOOP-1857
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1857
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.14.0
>            Reporter: Amareshwari Sri Ramadasu
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.15.0
>
>
> This basically is for providing a better user interface for debugging failed
> jobs. Today we see stack traces for failed tasks on the job ui if the job
> happened to be a Java MR job. For non-Java jobs like Streaming, Pipes, the
> diagnostic info on the job UI is not helpful enough to debug what might have
> gone wrong. They are usually framework traces and not app traces.
> We want to be able to provide a facility, via user-provided scripts, for doing
> post-processing on task logs, input, output, etc. There should be some default
> scripts like running core dumps under gdb for locating illegal instructions,
> the last few lines from stderr, etc.  These outputs could be sent to the
> tasktracker and in turn to the jobtracker which would then display it on the
> job UI on demand.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.