You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Deals Collect <de...@gmail.com> on 2013/04/09 01:49:08 UTC

Permgen size keeps increasing

Hi all,

I have a site crawling the deals every days. I have a problem with permgen
size. It just keeps increasing after each crawling. That's why I got the
"Out of memory" issue. As I know, Permgen heap just stores the class
definition, metadata... I'm not sure why it keeps increasing like this...
I'm using Nutch 1.4 and Solr 3.1. Here is the link that you can see the
permgen size: http://www.dealscollect.com.au/permgen  It is the last one:

Name: PS Perm Gen
Type: Non-heap memory
Max Usage: 268435456
Used: 75770136

Many thanks,
Vu

Re: Permgen size keeps increasing

Posted by Sebastian Nagel <wa...@googlemail.com>.
Hi,

> It just keeps increasing after each crawling.
What does this precisely mean?
(a) Are you running one crawl process with many cycles (depth)
    by launching "bin/nutch crawl" (org.apache.nutch.crawl.Crawl)
(b) or in separate steps (inject, generate, fetch, parse, updatedb, ...)?

For (a) see NUTCH-356 and NUTCH-1087 which may cause the problem.

Sebastian


On 04/09/2013 01:49 AM, Deals Collect wrote:
> Hi all,
> 
> I have a site crawling the deals every days. I have a problem with permgen
> size. It just keeps increasing after each crawling. That's why I got the
> "Out of memory" issue. As I know, Permgen heap just stores the class
> definition, metadata... I'm not sure why it keeps increasing like this...
> I'm using Nutch 1.4 and Solr 3.1. Here is the link that you can see the
> permgen size: http://www.dealscollect.com.au/permgen  It is the last one:
> 
> Name: PS Perm Gen
> Type: Non-heap memory
> Max Usage: 268435456
> Used: 75770136
> 
> Many thanks,
> Vu
>