You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by Paul van Brouwershaven <pa...@vanbrouwershaven.com> on 2005/09/28 09:14:04 UTC

Fetcher Speed with Threads

Hello,

I'm tuning my crawker but have some problems with the fetcher.

I have tried some different configurations but I don't get the fetcher 
above the 100k of traffic.

This command fetches the information with 100k:
$NUTCHCMD fetch -threads 50 $segment

But this command also ??
$NUTCHCMD fetch -threads 500 $segment

When I start some fetches (on diffrent segments) on the same time I will 
get the traffic of "number of fetchers" x 100k.

Who can help me to get this fetcher at some Mbps speed?

Regards,

Paul