You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Andy Morris <an...@woodward.edu> on 2006/01/26 11:54:41 UTC

After the crawl

 After running the initial crawl what command do I need to run on a
weekly or daily basis to keep my indexes up to date...is it "fetch"

Andy

Re: After the crawl

Posted by Gal Nitzan <gn...@usa.net>.
Hi Andy,

There are a few commands for keeping your index up 2 date.

after 30 days (default, can be changed in nutch-site.xml)

generate - generate a fetch list to be crawled in a new segments
fetch - will fetch the pages listed in the fetchlist
updatedb - update the web db with the list of pages and links found in
the fetch process
invertlinks - update the link db
index - update your indexes
dedup - remove duplicates
merge - merge your indexes




On Thu, 2006-01-26 at 05:54 -0500, Andy Morris wrote:
>  After running the initial crawl what command do I need to run on a
> weekly or daily basis to keep my indexes up to date...is it "fetch"
> 
> Andy
>