You are viewing a plain text version of this content. The canonical link for it is here.

Posted to agent@nutch.apache.org by 陈俊龙 <ch...@gmail.com> on 2010/01/16 03:08:23 UTC

fetch2 slow problem

Hi，everyone!
     I use nutch 1.0 to crawl 10M URLs,I found the fetch slow down when the
urls below almost 3000 or 4000 urls. and I change the threads to different
numbers(1800,1000,600),but make no difference.
     That look like, at the begining or when the urls number is large ,the
speed is very fast.but in the en more and more slow.

-- 
Best wishes！
chenli