Store Janitor Settings

[Gordon Anderson <go...@3months.com>]

Hi everyone,

After a bit of experimentation I have found that setting the StoreJanitor
max to around 600M and leaving the heap size at 1000M seems to work well.
These are the settings I used.



  <store-janitor logger="core.store.janitor">
                <!-- How much free memory shall be available in the jvm -->
                <parameter name="freememory" value="2048000"/>
                <!-- Indicates the limit of the jvm memory consumption. The
default maxheapsize for Sun's JVM is (almost) 64Mb -->
                <parameter name="heapsize" value="66600000"/>
                <!-- How often shall the cleanup thread check memory -->
                <parameter name="cleanupthreadinterval" value="10"/>
                <!-- Experimental adaptive algorithm for cleanup interval
     <parameter name="adaptivethreadinterval" value="true"/>
     -->
                <!-- Indicates the thread priority of the cleanup thread -->
                <parameter name="threadpriority" value="5"/>
                <!-- How much percent of the elements of each registered
Store
          shall be removed when low on memory. Default 10% -->
                <parameter name="percent_to_free" value="10"/>
                <!-- Invoke the garbage collector when low memory is reached
-->
                <parameter name="invokegc" value="false"/>
        </store-janitor>


  A stress test of several hours duration showed no memory leakage (after
approx 150000 hits) and acceptable response time.  It seems that Cocoon gets
into problems when it start to run out of memory and garbage collects all
the time - the logic for setting the StoreJanitor threshold way below the
maximum heap size is to try and prevent that very situation.  The rest of
the settings were as previously detailed in an email sent last week, which I
have appended below.

Hope that helps!

Cheers

Gordon

Gordon Anderson
3months.com
Wellington
New Zealand


hi

I've been looking into variations of performance with different versions of 
Tomcat to serve a website of approximately 6000 URLs using Cocoon 
2.1.4.  Performance was noticeably slower using Tomcat 4, the reason being 
garbage collection.

To turn garbage collection on using the Sun JVM, append

-Xloggc:/opt/tomcat/logs/garbage.log

to the CATALINA_OPTS variable, where /opt/tomcat is the location of your 
Tomcat instance.

Crawling the site using the free Xenu crawler tool, performance under 
Tomcat 4.1.27 degraded significantly after around 1000 URLs.  Examining the 
logs, the reason was soon obvious.

1334.524: [Full GC 518449K->518079K(518464K), 2.5774490 secs]
1337.102: [Full GC 518079K->518079K(518464K), 2.5330770 secs]
1339.654: [Full GC 518458K->518188K(518464K), 2.5701920 secs]
1342.224: [Full GC 518188K->518179K(518464K), 2.5360680 secs]
1344.802: [Full GC 518456K->518233K(518464K), 2.5695450 secs]
1347.371: [Full GC 518233K->518214K(518464K), 3.2225810 secs]
1350.610: [Full GC 518403K->518180K(518464K), 2.5540920 secs]
1353.164: [Full GC 518180K->518178K(518464K), 2.5558220 secs]


It was spending all its time garbage collecting!



Using Tomcat 5.0.19, after approx 30000 hits on the site, the log is 
certainly more pleasing:
13435.211: [GC 749150K->686258K(765332K), 0.1123580 secs]
13442.140: [GC 755121K->692181K(765332K), 0.1140720 secs]
13449.131: [GC 761045K->700089K(769044K), 0.1324210 secs]
13449.265: [Full GC 700089K->233002K(769044K), 1.8186920 secs]
13460.298: [GC 301859K->245577K(765332K), 0.1021830 secs]
13468.507: [GC 314440K->254022K(765332K), 0.1221460 secs]
13476.208: [GC 322886K->259107K(765332K), 0.1037680 secs]
13482.677: [GC 327971K->267097K(765332K), 0.1375070 secs]
13492.732: [GC 335961K->274713K(765332K), 0.1315550 secs]
13500.313: [GC 343577K->279775K(765332K), 0.1020140 secs]
13508.526: [GC 348634K->286971K(765332K), 0.1172210 secs]
13517.606: [GC 355831K->292747K(765332K), 0.1061540 secs]


As you can see the Full GC (Full Garbage Collect) is happening 
sporadically, thus server performance is improved.


It should be noted that exactly the same WAR file and tomcatd / 
CATALINA_OPTS were used for both versions of Tomcat.


The parameters I have changed from the default installation:

garbage collection logging as above
set max heap size to 1000M
made NO CHANGES to the store janitor parameters
set all pool-max values for the objects I am using to 1024

Use Tomcat 5.0.19, JDK 1.4.2
CIncludes cached for 12 hours
Apache serving static files


As a little side note after the crawl had finished, I saw 700M of heap 
space being reclaimed by a garbage collect :)  This would indicate that the 
memory issues that were inherent with 2.1.2 have disappeared.


I hope the above is of interest to those having performance problems


Cheers

Gordon

Gordon Anderson
3months.com




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org