You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@nutch.apache.org by "Markus Jelsma (JIRA)" <ji...@apache.org> on 2016/02/23 12:57:18 UTC

[jira] [Resolved] (NUTCH-2221) Introduce db.ignore.internal.links to FetcherThread

     [ https://issues.apache.org/jira/browse/NUTCH-2221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Markus Jelsma resolved NUTCH-2221.
----------------------------------
    Resolution: Fixed
      Assignee: Markus Jelsma

> Introduce db.ignore.internal.links to FetcherThread
> ---------------------------------------------------
>
>                 Key: NUTCH-2221
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2221
>             Project: Nutch
>          Issue Type: Improvement
>          Components: fetcher
>    Affects Versions: 1.11
>            Reporter: Markus Jelsma
>            Assignee: Markus Jelsma
>             Fix For: 1.12
>
>         Attachments: NUTCH-2216-NUTCH-2220-NUTCH-2221.patch, NUTCH-2221.patch, NUTCH-2221.patch
>
>
> FetcherThread has support for db.ignore.external.links. In config you can find db.ignore.internal.links as well, but it only operates on LinkDB, which is confusing. This patch will introduce db.ignore.internal.links to FetcherThread, similar to db.ignore.external.links. With both parameter set to true you can limit the crawl to the injected seed list.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)