You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@nutch.apache.org by "Rod Taylor (JIRA)" <ji...@apache.org> on 2006/03/30 18:21:27 UTC

[jira] Updated: (NUTCH-242) Add optional -urlFiltering to updatedb

     [ http://issues.apache.org/jira/browse/NUTCH-242?page=all ]

Rod Taylor updated NUTCH-242:
-----------------------------

    Attachment: nutch_crawldb_filtering.patch

> Add optional -urlFiltering to updatedb
> --------------------------------------
>
>          Key: NUTCH-242
>          URL: http://issues.apache.org/jira/browse/NUTCH-242
>      Project: Nutch
>         Type: New Feature
>     Versions: 0.8-dev
>     Reporter: Rod Taylor
>  Attachments: nutch_crawldb_filtering.patch
>
> Allow filtering the URLs completely out of the database during an updatedb run. This allows the regex-urlfilter.xml rules to be changed and the non-matching entries to be expunged from the database.
> Merging the functionality with updatedb was done for efficiency reasons since this eats up CPU time only where a separate job would use IO and CPU time.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira