[jira] Commented: (NUTCH-49) Flag for generate to fetch only new pages to complement the -refetchonly flag

["Alex McLintock (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/NUTCH-49?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12883488#action_12883488 ] 

Alex McLintock commented on NUTCH-49:
-------------------------------------

Is there still interest in this? I'd like to produce a new patch for this if anyone wants it. It sounds like a good idea to me.

> Flag for generate to fetch only new pages to complement the -refetchonly flag
> -----------------------------------------------------------------------------
>
>                 Key: NUTCH-49
>                 URL: https://issues.apache.org/jira/browse/NUTCH-49
>             Project: Nutch
>          Issue Type: New Feature
>          Components: fetcher
>            Reporter: Luke Baker
>            Priority: Minor
>         Attachments: fetchnewonly.patch
>
>
> It would be useful, especially for research/testing purposes, to have a flag for the FetchListTool that make sure to only include URLs in the fetchlist that have not already been fetched (according to the information from the webdb that you're generating the fetchlist from).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.