[jira] [Commented] (NUTCH-1302) nutchgora job failures should be noticed by submitter

["Ferdy Galema (Commented) (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NUTCH-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223231#comment-13223231 ] 

Ferdy Galema commented on NUTCH-1302:
-------------------------------------

NutchJob is a nice wrapper of Hadoop's Job, so that I could override Job.waitForCompletion(verbose) in order to throw an exception whenever a job fails. (A job fails typically whenever one or more tasks have failed repeatedly.)

Because some people may be reliant upon silent job failures (could not think of why, but just in case), I created an expert option that disables this behaviour, namely "fail.on.job.failure". When this property is set to false, jobs will fail silently like they did before.
                
> nutchgora job failures should be noticed by submitter
> -----------------------------------------------------
>
>                 Key: NUTCH-1302
>                 URL: https://issues.apache.org/jira/browse/NUTCH-1302
>             Project: Nutch
>          Issue Type: Bug
>            Reporter: Ferdy Galema
>            Assignee: Ferdy Galema
>
> I stumbled upon an issue where crawling seem to go right, only to notice much later on that jobs actually failed as a whole.
> This is caused because for most jobs that are submitted, Nutchgora does not check the 'succeeded' boolean that is returned. This should be done and acted upon appropriately. (Either throwing an exception or returning non-zero exit codes).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira