You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by "Ferdy Galema (Created) (JIRA)" <ji...@apache.org> on 2012/03/06 13:32:57 UTC
[jira] [Created] (NUTCH-1302) nutchgora job failures should be
noticed by submitter
nutchgora job failures should be noticed by submitter
-----------------------------------------------------
Key: NUTCH-1302
URL: https://issues.apache.org/jira/browse/NUTCH-1302
Project: Nutch
Issue Type: Bug
Reporter: Ferdy Galema
Assignee: Ferdy Galema
I stumbled upon an issue where crawling seem to go right, only to notice much later on that jobs actually failed as a whole.
This is caused because for most jobs that are submitted, Nutchgora does not check the 'succeeded' boolean that is returned. This should be done and acted upon appropriately. (Either throwing an exception or returning non-zero exit codes).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1302) nutchgora job failures should be
noticed by submitter
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223955#comment-13223955 ]
Hudson commented on NUTCH-1302:
-------------------------------
Integrated in Nutch-nutchgora #185 (See [https://builds.apache.org/job/Nutch-nutchgora/185/])
NUTCH-1302 nutchgora job failures should be noticed by submitter (Revision 1297462)
Result = SUCCESS
ferdy :
Files :
* /nutch/branches/nutchgora/CHANGES.txt
* /nutch/branches/nutchgora/src/java/org/apache/nutch/util/NutchJob.java
> nutchgora job failures should be noticed by submitter
> -----------------------------------------------------
>
> Key: NUTCH-1302
> URL: https://issues.apache.org/jira/browse/NUTCH-1302
> Project: Nutch
> Issue Type: Bug
> Reporter: Ferdy Galema
> Assignee: Ferdy Galema
> Attachments: NUTCH-1302.patch
>
>
> I stumbled upon an issue where crawling seem to go right, only to notice much later on that jobs actually failed as a whole.
> This is caused because for most jobs that are submitted, Nutchgora does not check the 'succeeded' boolean that is returned. This should be done and acted upon appropriately. (Either throwing an exception or returning non-zero exit codes).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (NUTCH-1302) nutchgora job failures should be
noticed by submitter
Posted by "Ferdy Galema (Closed) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema closed NUTCH-1302.
-------------------------------
Resolution: Fixed
> nutchgora job failures should be noticed by submitter
> -----------------------------------------------------
>
> Key: NUTCH-1302
> URL: https://issues.apache.org/jira/browse/NUTCH-1302
> Project: Nutch
> Issue Type: Bug
> Reporter: Ferdy Galema
> Assignee: Ferdy Galema
> Attachments: NUTCH-1302.patch
>
>
> I stumbled upon an issue where crawling seem to go right, only to notice much later on that jobs actually failed as a whole.
> This is caused because for most jobs that are submitted, Nutchgora does not check the 'succeeded' boolean that is returned. This should be done and acted upon appropriately. (Either throwing an exception or returning non-zero exit codes).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1302) nutchgora job failures should be
noticed by submitter
Posted by "Ferdy Galema (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated NUTCH-1302:
--------------------------------
Attachment: NUTCH-1302.patch
> nutchgora job failures should be noticed by submitter
> -----------------------------------------------------
>
> Key: NUTCH-1302
> URL: https://issues.apache.org/jira/browse/NUTCH-1302
> Project: Nutch
> Issue Type: Bug
> Reporter: Ferdy Galema
> Assignee: Ferdy Galema
> Attachments: NUTCH-1302.patch
>
>
> I stumbled upon an issue where crawling seem to go right, only to notice much later on that jobs actually failed as a whole.
> This is caused because for most jobs that are submitted, Nutchgora does not check the 'succeeded' boolean that is returned. This should be done and acted upon appropriately. (Either throwing an exception or returning non-zero exit codes).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1302) nutchgora job failures should be
noticed by submitter
Posted by "Ferdy Galema (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/NUTCH-1302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13223231#comment-13223231 ]
Ferdy Galema commented on NUTCH-1302:
-------------------------------------
NutchJob is a nice wrapper of Hadoop's Job, so that I could override Job.waitForCompletion(verbose) in order to throw an exception whenever a job fails. (A job fails typically whenever one or more tasks have failed repeatedly.)
Because some people may be reliant upon silent job failures (could not think of why, but just in case), I created an expert option that disables this behaviour, namely "fail.on.job.failure". When this property is set to false, jobs will fail silently like they did before.
> nutchgora job failures should be noticed by submitter
> -----------------------------------------------------
>
> Key: NUTCH-1302
> URL: https://issues.apache.org/jira/browse/NUTCH-1302
> Project: Nutch
> Issue Type: Bug
> Reporter: Ferdy Galema
> Assignee: Ferdy Galema
>
> I stumbled upon an issue where crawling seem to go right, only to notice much later on that jobs actually failed as a whole.
> This is caused because for most jobs that are submitted, Nutchgora does not check the 'succeeded' boolean that is returned. This should be done and acted upon appropriately. (Either throwing an exception or returning non-zero exit codes).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira