RE: Large documents - OutOfMemoryError with Tomcat but not Jetty! !!

[Ph...@empolis.com]

Jan,

Thanks for the info.

The server itself has two gigs of RAM and I tried up to 500MB for
'CATALINA_OPTS=-Xmx'

I am using /startup.sh.

But the bit I don't understand, and maybe someone who knows how the version
of Jetty that comes with Cocoon was configured, is why it would work with
that version of Jetty. Has some arbitrarily large amount of memory been
allocated that could account for most usage patterns???


Regards

Phil Fennell


-----Original Message-----
From: Jan Hoskens [mailto:jh@schaubroeck.be] 
Sent: 01 June 2004 14:21
To: users@cocoon.apache.org
Subject: Re: Large documents - OutOfMemoryError with Tomcat but not Jetty!!!

I'm no expert in this, but what's your exact memory amount you've tried? I'm
using CATALINA_OPTS=-Xmx384M and that works fine with me, although my pdf
documents aren't that big. I also had a small problem that the tomcat that I
installed: it created a shortcut in my program files that did not start with
the correct parameters, you do need to start with the startup.bat /sh in the
bin directory to have that memory parameter in use.

I also switched to running my pdf stuff in a thread because fop takes a lot
of time.
When the xml doc is ready I call a thread in my flow that handles the fop
part while I send the user a page that tells him his request is handled and
that a link will appear on some page when the pdf is ready. This may give
some ideas, because you're mentioning the twenty minute waiting period.


Kind Regards,
Jan

----- Original Message -----
From: <Ph...@empolis.com>
To: <us...@cocoon.apache.org>
Sent: Tuesday, June 01, 2004 2:18 PM
Subject: Large documents - OutOfMemoryError with Tomcat but not Jetty!!!


> I have a problem when using Cocoon to aggregate large amounts of content
> that is to be serialised as PDF.
>
> I have a set of pipelines that build a XML document that contains a set of
> xincludes which is then transformed using the xinclude transformer before
> conversion into XSL-FO and final serialisation to PDF. Using all the
> standard transformers - nothing extra, XML, XSLT, Xinclude but no XSP and
> the like.
>
> If I run this using Tomcat it works for small document sets (29) amounting
> to 218KB of XML and 43KB of PDF.
>
> If I run this using Tomcat for a large document set (1044) amounting to
> 3.5MB of XML and 2.7MB of PDF then I get OutOfMemoryError messages in the
> logs (cocoon and tomcat) - see below.
>
> If I install a full version of Jetty I get outOfMemory errors too.
>
> However, if I run the same app within the version of Jetty that comes with
> Cocoon there is no problem!!!!
>
>
> *** Can anyone explain why the Jetty that comes with cocoon works while
> Tomcat does not??? ****
>
>
> I have include system config's and log extracts from both cocoon and
tomcat
> at the end of this e-mail.
>
>
> I have tried all the various tricks with changing memory settings for JVM
> like:
>
> CATALINA_OPTS=" -Xms128m -Xmx128m "
>
> But with no success. No out of memory errors, it just never finished after
> 20 minutes waiting!
>
> I have also ensured that I use the FOP tuning tips to ensure best
recycling
> of memory.
>
>
>
>
>
>
> Server Configuration
> --------------------
>
> OS: Red Hat Linux - Advanced Server version 2.1
>
> HTTP Server: Apache httpd-2.0.48
>
> VM: Java 1.4.2_02-b03
>
> AppServer: Tomcat 4.1.30
>
> App: Cocoon 2.1.2
>
>
>
>
> cocoon/WEB-INF/logs/core.log
> ----------------------------
>
> WARN    (2004-06-01) 10:49.01:190   [core.manager]
>
(/cocoon/RIU/Aggregate/Content/Documents/World/Former_Soviet_Union/Russia/Ru
> ssia.pdf) TP-Processor7/ExcaliburComponentManager: Attempted to release a
> org.apache.cocoon.components.treeprocessor.sitemap.ComponentsSelector but
> its handler could not be located.
> ERROR   (2004-06-01) 10:50.44:818   [core.store.janitor] (Unknown-URI)
> Unknown-thread/StoreJanitorImpl: OutOfMemoryError in freeMemory()
> ERROR   (2004-06-01) 10:51.07:636   [core.store.janitor] (Unknown-URI)
> Unknown-thread/StoreJanitorImpl: OutOfMemoryError in freeMemory()
> WARN    (2004-06-01) 10:51.16:127   [core.manager]
>
(/cocoon/RIU/Aggregate/Content/Documents/World/Former_Soviet_Union/Russia/Ru
> ssia.pdf) TP-Processor7/ExcaliburComponentManager: Attempted to release a
> org.apache.cocoon.components.treeprocessor.sitemap.ComponentsSelector but
> its handler could not be located.
> ERROR   (2004-06-01) 10:52.34:308   [core.store.janitor] (Unknown-URI)
> Unknown-thread/StoreJanitorImpl: OutOfMemoryError in freeMemory()
>
>
> cocoon/WEB-INF/logs/error.log
> -----------------------------
>
> ERROR   (2004-06-01) 10:50.44:819   [core.store.janitor] (Unknown-URI)
> Unknown-thread/StoreJanitorImpl: OutOfMemoryError in freeMemory()
> ERROR   (2004-06-01) 10:51.09:073   [core.store.janitor] (Unknown-URI)
> Unknown-thread/StoreJanitorImpl: OutOfMemoryError in freeMemory()
> ERROR   (2004-06-01) 10:52.34:309   [core.store.janitor] (Unknown-URI)
> Unknown-thread/StoreJanitorImpl: OutOfMemoryError in freeMemory()
>
>
> Tomcat/logs/localhost_log.2004-06-01.txt
> ----------------------------------------
>
> 2004-06-01 10:51:15 StandardWrapperValve[Cocoon]: Servlet.service() for
> servlet Cocoon threw exception
> javax.servlet.ServletException: Servlet execution threw an exception
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:269)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:193)
> at
>
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
> va:256)
> at
>
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
> eNext(StandardPipeline.java:643)
> at
>
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
> at
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> at
>
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
> va:191)
> at
>
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
> eNext(StandardPipeline.java:643)
> at
>
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
> at
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> at
> org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2422)
> at
>
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180
> )
> at
>
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
> eNext(StandardPipeline.java:643)
> at
>
org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.
> java:171)
> at
>
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
> eNext(StandardPipeline.java:641)
> at
>
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:163
> )
> at
>
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
> eNext(StandardPipeline.java:641)
> at
>
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
> at
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> at
>
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
> :174)
> at
>
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
> eNext(StandardPipeline.java:643)
> at
>
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
> at
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> at
> org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:199)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:828)
> at
>
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConne
> ction(Http11Protocol.java:700)
> at
> org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:584)
> at
>
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.jav
> a:683)
> at java.lang.Thread.run(Thread.java:534)
> ----- Root Cause -----
> java.lang.OutOfMemoryError
>
> 2004-06-01 10:52:37 StandardWrapperValve[Cocoon]: Servlet.service() for
> servlet Cocoon threw exception
> javax.servlet.ServletException: Servlet execution threw an exception
> at
>
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Application
> FilterChain.java:269)
> at
>
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterCh
> ain.java:193)
> at
>
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.ja
> va:256)
> at
>
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
> eNext(StandardPipeline.java:643)
> at
>
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
> at
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> at
>
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.ja
> va:191)
> at
>
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
> eNext(StandardPipeline.java:643)
> at
>
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
> at
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> at
> org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2422)
> at
>
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:180
> )
> at
>
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
> eNext(StandardPipeline.java:643)
> at
>
org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.
> java:171)
> at
>
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
> eNext(StandardPipeline.java:641)
> at
>
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:163
> )
> at
>
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
> eNext(StandardPipeline.java:641)
> at
>
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
> at
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> at
>
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java
> :174)
> at
>
org.apache.catalina.core.StandardPipeline$StandardPipelineValveContext.invok
> eNext(StandardPipeline.java:643)
> at
>
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:480)
> at
> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:995)
> at
> org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:199)
> at
> org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:828)
> at
>
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.processConne
> ction(Http11Protocol.java:700)
> at
> org.apache.tomcat.util.net.TcpWorkerThread.runIt(PoolTcpEndpoint.java:584)
> at
>
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.jav
> a:683)
> at java.lang.Thread.run(Thread.java:534)
> ----- Root Cause -----
> java.lang.OutOfMemoryError
>
>
>
> Regards
>
> Philip A. R. Fennell
> Web Developer
> ------------------------------------------------------------------
> empolis UK Ltd.
> part of arvato: a Bertelsmann company
>
> Unit B, The Dorcan Complex,
> Faraday Road,
> Swindon,
> Wiltshire.
> SN3 5HQ
> United Kingdom
> http://www.empolis.co.uk
>
> e-mail: pf@empolis.co.uk
> Tel: +44 (0)1793 485465
> Fax: +44 (0)1793 485451
>
> Reg No: 3831777 VAT No: GB 718 2715 32
> ------------------------------------------------------------------
> The information in this email is confidential and may be
> legally privileged. It is intended solely for the addressee.
> Access to this email by anyone else is unauthorised.
> If you are not the intended recipient, any disclosure,
> copying, distribution or any action taken or omitted to be
> taken in reliance on it, is prohibited and may be unlawful.
> If you have received this communication in error please
> return it to the sender, then delete and destroy any copies of it.
> ------------------------------------------------------------------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
> For additional commands, e-mail: users-help@cocoon.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org