Any Special memory management measures when searching large nutch indexes with Tomcat?

[Vijay <vi...@cs.stanford.edu>]

Hi folks,

         I am enabling searching of a large nutch index with a few
million pages through a jsp page within a tomcat container. I find
that with some frequency of usage, the memory consumption of tomcat
seems to keep going up. I can sometimes visibly see tomcat's memory
usage go up by 50-100MB for some nutch queries, and it does not seem
to revert back quickly. This in turn over time eats up all the
allocated main memory (2.5GB in my case) and leads to out of memory
errors when tomcat subsequently calls nutch for serving a search
result.

          My questions are:
1. Is there something that needs to be done to make tomcat let go of
the extra memory immediately after serving a search result.

2. Can I do anything more general to put the java garbage collector on
steroids and make it remove junk objects, instead of doing it somewhat
lazily, particularly when the available memory is running out. Most of
the stuff I can find on the web relating to garbage collection for
servers tries to keep the amount of garbage collection minimal, so as
to not cause temporary slowness. My problem is the very opposite where
I think it would help if the garbage collector swept through memory at
a much faster rate.



Thanks,
Vijay