[jira] [Commented] (NUTCH-2242) lastModified not always set

["Sebastian Nagel (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NUTCH-2242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210136#comment-15210136 ] 

Sebastian Nagel commented on NUTCH-2242:
----------------------------------------

Hi Jurian, thanks for reporting this problem. This is part of a problem reported November last year, see NUTCH-2164 and the referenced thread on the user mailing list.
Thanks, for the patch. I've just 3 days prepared one for NUTCH-2164 on [github|https://github.com/apache/nutch/compare/master...sebastian-nagel:NUTCH-2164]. I'll merge both and resolve both issues within the next days. One question: why should the modified time also be set in CrawlDbReducer.reduce()? Shouldn't it be sufficient to do this once in the FetchSchedule implementation which is a customizable and pluggable.

> lastModified not always set
> ---------------------------
>
>                 Key: NUTCH-2242
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2242
>             Project: Nutch
>          Issue Type: Bug
>          Components: crawldb
>    Affects Versions: 1.11
>            Reporter: Jurian Broertjes
>            Priority: Minor
>         Attachments: NUTCH-2242.patch
>
>
> I observed two issues:
> - When using the DefaultFetchSchedule, CrawlDatum's modifiedTime field is not updated on the first successful fetch. 
> - When a document modification is detected (protocol- or signature-wise), the modifiedTime isn't updated
> I can provide a patch later today.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)