LinkDb creation is Too slow

[hareesh <ha...@hotmail.com>]

Hi,

I am facing a problem now. I will tell you the problem in detail.I have
crawled near about 1.5 Million urls.Every thing (Crawling and  Update Db)
was fine . Once this process was done. I started the invert link process. It
worked fine and created the linkdb. After that I was having some problem
with my Indexing plug-in. I did some code modifications to my plug-in and
every thing went fine till that.
After that i thought of indexing another segment. for that first I deleted
the linkdb already there and tried to create the linkdb from the new
segment. The problem is that the linkdb is taking too long its going in
hours.. now its almost 8 Hrs that have started the process. There is no
exception nothing. As I haven't done any modifications to the nutch source I
believe the problem is not because of that.

Present Segment Details:

No of URL in the segment : 80,000


I'm using a cluster of 4 with configuration of 2Tb hard drive and 4GB Ram.
All the systems except one is  having enough disk space too. one of the
system from the cluster is having 48GB only as free.

Can any one help me with there comments. Thanks in advance.


-- 
View this message in context: http://lucene.472066.n3.nabble.com/LinkDb-creation-is-Too-slow-tp888368p888368.html
Sent from the Nutch - User mailing list archive at Nabble.com.