You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by Bostjan <bg...@siol.net> on 2006/02/14 09:10:03 UTC

Max pages in crawl cycle

Hi,

I'm using nutch 0.7.

Is it possible to crawl only certain number of pages in single crawl cycle 
(depth)?  I looked at FetchList Tool class and I think it would be nice that 
emitFetchList method had a piece of code in its main loop that woud look 
something like this

    if (count > MAX_PAGES_IN_CRAWL_CYCLE) {
        break;
    }

Thanks,
Bostjan