You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by Doğacan Güney <do...@gmail.com> on 2007/09/11 21:18:59 UTC

Re: Why 'nutch generate' is ignoring my argument of -numFetchers

Hi,

On 9/11/07, Jenny LIU <je...@yahoo.com> wrote:
> When I do:
>   nutch generate crawl/db crawl/segments -numFetchers 30 -topN 5000
>   I was trying to get 30 partitions of segments (30 subdirs in segments dir generated)
>   but instead, it give me:
>   Generator: jobtracker is 'local', generating exactly one partition
>   so only one partition generated (one subdir in segments dir generated)

You can only generate one partition when MR is in local mode. That is
just the way things are with hadoop.

>
>   that one partition seems too big for me to fetch it later on,
>
>   Why 'nutch generate'  is ignoring my argument of -numFetchers?
>
>   Thank you.
>
>   Jenny
>
>
>
>
> ---------------------------------
> Got a little couch potato?
> Check out fun summer activities for kids.


-- 
Doğacan Güney