Re: Fetcher hanging temporarily on "deflateBytes" method

[Daniel Varela Santoalla <dv...@ecmwf.int>]

Hi Denis

I thought that the problem was in that method is because I saw with the 
thread dumps and Eclipse's debugger that the thread was blocked there. 
Were you seeing the same thing?

BTW, if I run the crawl in a Pentium IV machine, I see 0% CPU usage 
during the "blocked time", but if I do it in an x86_64, I see 100%. 
Interesting.

For me this is a very serious problem, because the indexing is taking 
ages to finish. With java1.4 the problem is the same, I'm afraid.

Thanks for your help.

Daniel


Dennis Kubes wrote:
> When I was researching this issue I first thought it was the 
> deflateBytes method as well but when I changed things in the code the 
> problem persisted until I changed the regex filter.  Maybe your problem 
> actually is in the deflate bytes method.  The forum I was talking about 
> earlier was nutch-user but if you don't have the regex then those posts 
> wouldn't help you.  Here is the text of a previous conversation I had 
> about this with Stefan. ------
> I have this suspicion that the inflater class in Java 1.5 is causing 
> some problems with spinning but I can't prove it.  We are using about 
> the same java and linux versions.
> 


-- 

Daniel Varela Santoalla
European Centre for Medium-Range Weather Forecasts (ECMWF) 
(http://www.ecmwf.int)
Email: dvarela@ecmwf.int    Telephone: (+44) 118 9499608