You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by "George A. Papayiannis" <pa...@gmail.com> on 2005/07/27 23:03:41 UTC
updating a crawl
Hi,
I have a list of URLS I'm crawling, and I want to crawl the
frequently. I crawl the list initially using ./nutch crawl urls -dir
testcrawl -depth 1 and it works fine.
My question is, what is the best way to update directory testcrawl?
>From what it seems, I can't directly write to testcrawl during the
crawl. I have to crawl in a separate directory and then merge them?
Is this correct?
Thanks for the help,
George