You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Stephan Michels <st...@apache.org> on 2003/03/06 16:10:35 UTC
Performance leak
Hi,
I do some profiling today, and wondering why the XSLT transformer
burns performance like hell in setup(). After some debugging I noticed
that the XSLT transformer create a complete new template/handler for
each call of setup(), that because the parameter 'use-store' in
cocoon.xconf is set to 'false' per default.
After setting this parameter to 'true', the performance increased
dramatically.
So, is there a reason, why this parameter is set to 'false'? When
not, I would change it to 'true'.
Stephan.
_______________________________________________________________________
Stephan Michels EMail: stephan@apache.org
ICQ: 115535699 Tel: +49-030-314-21583
----+----|----+----|----+----|----+----|----+----|----+----|----+----|-|
Re: XSLTProcessor and Store was: Performance leak
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Stephan Michels wrote:
>On Fri, 7 Mar 2003, Sylvain Wallez wrote:
>
>
>
>>>Yesterday I measured a performance enhancement by the factor 20x .
>>>Before each request took 2000-5000ms, which was untenable. After
>>>setting this option, each request took 50-100ms(cached).
>>>
>>>Perhaps you have an idea, where the problem lie on?
>>>
>>>
>>>
>>The problem, I think, comes from the fact that XSLTProcessor uses the
>>transient store to store Templates, and that the <transient-store> in
>>cocoon.xconf has the <parameter name="use-persistent-cache" value="true"/>.
>>
>>So we actually have a transient store that persists its contents to disk !!!
>>
>>
>
>This is okay, if it's only persists in dispose().
>
Unfortunately, it also persists on free() when the size exceeds the MRU
limit (see org.apache.excalibur.store.impl.MRUMemoryStore)
Moreover, I just realized (unless I'm missing something) that our
<persistent-store> only read/writes to disk without an in-memory MRU
front-end. This may also explain some poor performance.
>But a simple (object instanceof java.io.Serializable) due to
>checkSerializable() seems to be not enough to test if a object
>is serializable.
>
Yes, this isn't enough since the whole object graph starting at the
serialized object must also be Serializable.
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
Re: XSLTProcessor and Store was: Performance leak
Posted by Stephan Michels <st...@apache.org>.
On Fri, 7 Mar 2003, Sylvain Wallez wrote:
> Stephan Michels wrote:
>
> >On Thu, 6 Mar 2003, Stephan Michels wrote:
> >
> >>On Thu, 6 Mar 2003, Carsten Ziegeler wrote:
> >>
> >>>Stephan Michels wrote:
> >>>
> >>>>I do some profiling today, and wondering why the XSLT transformer
> >>>>burns performance like hell in setup(). After some debugging I noticed
> >>>>that the XSLT transformer create a complete new template/handler for
> >>>>each call of setup(), that because the parameter 'use-store' in
> >>>>cocoon.xconf is set to 'false' per default.
> >>>>
> >>>>After setting this parameter to 'true', the performance increased
> >>>>dramatically.
> >>>>
> >>>>So, is there a reason, why this parameter is set to 'false'? When
> >>>>not, I would change it to 'true'.
> >>>>
> >>>There was a reason: a memory leak :) So you can change from performance
> >>>to memory leak. This leak was at least in last Xalan versions, I never
> >>>tested if it is in the latest, so perhaps this is fixed now.
> >>>If not, you can run into serious memory problems.
> >>>
> >>Doh! ;-) I'll make some longtime stability tests, to verify this. The
> >>memory leak should also occur if you use the store, except if you forget
> >>to remove the entries. At the moment, the concept of the XSLTC(and other)
> >>doesn't make any sense, because you compile all used stylesheets for
> >>every request :-/
> >
> >I got NotSerializableException, if I stop the container. This remind
> >with my problems with LexicalTransformer. I solved this problem
> >using own writeObject() and readObject() methods, but the
> >XSLTProcessorImpl using Object[] instances. I also don't know why this
> >array isn't serializable, perhaps the Templates class isn't serializable.
> >
> >Yesterday I measured a performance enhancement by the factor 20x .
> >Before each request took 2000-5000ms, which was untenable. After
> >setting this option, each request took 50-100ms(cached).
> >
> >Perhaps you have an idea, where the problem lie on?
> >
>
> The problem, I think, comes from the fact that XSLTProcessor uses the
> transient store to store Templates, and that the <transient-store> in
> cocoon.xconf has the <parameter name="use-persistent-cache" value="true"/>.
>
> So we actually have a transient store that persists its contents to disk !!!
This is okay, if it's only persists in dispose().
But a simple (object instanceof java.io.Serializable) due to
checkSerializable() seems to be not enough to test if a object
is serializable.
:-/ Stephan Michels.
Re: XSLTProcessor and Store was: Performance leak
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Stephan Michels wrote:
>On Thu, 6 Mar 2003, Stephan Michels wrote:
>
>
>
>>On Thu, 6 Mar 2003, Carsten Ziegeler wrote:
>>
>>
>>
>>>Stephan Michels wrote:
>>>
>>>
>>>>I do some profiling today, and wondering why the XSLT transformer
>>>>burns performance like hell in setup(). After some debugging I noticed
>>>>that the XSLT transformer create a complete new template/handler for
>>>>each call of setup(), that because the parameter 'use-store' in
>>>>cocoon.xconf is set to 'false' per default.
>>>>
>>>>After setting this parameter to 'true', the performance increased
>>>>dramatically.
>>>>
>>>>So, is there a reason, why this parameter is set to 'false'? When
>>>>not, I would change it to 'true'.
>>>>
>>>>
>>>>
>>>There was a reason: a memory leak :) So you can change from performance
>>>to memory leak. This leak was at least in last Xalan versions, I never
>>>tested if it is in the latest, so perhaps this is fixed now.
>>>If not, you can run into serious memory problems.
>>>
>>>
>>Doh! ;-) I'll make some longtime stability tests, to verify this. The
>>memory leak should also occur if you use the store, except if you forget
>>to remove the entries. At the moment, the concept of the XSLTC(and other)
>>doesn't make any sense, because you compile all used stylesheets for
>>every request :-/
>>
>>
>
>I got NotSerializableException, if I stop the container. This remind
>with my problems with LexicalTransformer. I solved this problem
>using own writeObject() and readObject() methods, but the
>XSLTProcessorImpl using Object[] instances. I also don't know why this
>array isn't serializable, perhaps the Templates class isn't serializable.
>
>Yesterday I measured a performance enhancement by the factor 20x .
>Before each request took 2000-5000ms, which was untenable. After
>setting this option, each request took 50-100ms(cached).
>
>Perhaps you have an idea, where the problem lie on?
>
The problem, I think, comes from the fact that XSLTProcessor uses the
transient store to store Templates, and that the <transient-store> in
cocoon.xconf has the <parameter name="use-persistent-cache" value="true"/>.
So we actually have a transient store that persists its contents to disk !!!
Sylvain
--
Sylvain Wallez Anyware Technologies
http://www.apache.org/~sylvain http://www.anyware-tech.com
{ XML, Java, Cocoon, OpenSource }*{ Training, Consulting, Projects }
XSLTProcessor and Store was: Performance leak
Posted by Stephan Michels <st...@apache.org>.
On Thu, 6 Mar 2003, Stephan Michels wrote:
> On Thu, 6 Mar 2003, Carsten Ziegeler wrote:
>
> >
> > Stephan Michels wrote:
> > >
> > > I do some profiling today, and wondering why the XSLT transformer
> > > burns performance like hell in setup(). After some debugging I noticed
> > > that the XSLT transformer create a complete new template/handler for
> > > each call of setup(), that because the parameter 'use-store' in
> > > cocoon.xconf is set to 'false' per default.
> > >
> > > After setting this parameter to 'true', the performance increased
> > > dramatically.
> > >
> > > So, is there a reason, why this parameter is set to 'false'? When
> > > not, I would change it to 'true'.
> > >
> > There was a reason: a memory leak :) So you can change from performance
> > to memory leak. This leak was at least in last Xalan versions, I never
> > tested if it is in the latest, so perhaps this is fixed now.
> > If not, you can run into serious memory problems.
>
> Doh! ;-) I'll make some longtime stability tests, to verify this. The
> memory leak should also occur if you use the store, except if you forget
> to remove the entries. At the moment, the concept of the XSLTC(and other)
> doesn't make any sense, because you compile all used stylesheets for
> every request :-/
I got NotSerializableException, if I stop the container. This remind
with my problems with LexicalTransformer. I solved this problem
using own writeObject() and readObject() methods, but the
XSLTProcessorImpl using Object[] instances. I also don't know why this
array isn't serializable, perhaps the Templates class isn't serializable.
Yesterday I measured a performance enhancement by the factor 20x .
Before each request took 2000-5000ms, which was untenable. After
setting this option, each request took 50-100ms(cached).
Perhaps you have an idea, where the problem lie on?
Thanks, Stephan Michels.
ERROR (2003-03-07) 15:12.15:772 [core.store.transient] (Unknown-URI)
Unknown-thread/MRUMemoryStore: Error in dispose()
java.io.NotSerializableException:
org.apache.excalibur.xml.xslt.XSLTProcessorImpl
at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1054)
at
java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1330)
at
java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1302)
at
java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1245)
at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1052)
at
java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1224)
at
java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1050)
at
java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:278)
at
org.apache.excalibur.store.impl.AbstractFilesystemStore.serializeObject(AbstractFilesystemStore.java:479)
at
org.apache.excalibur.store.impl.AbstractFilesystemStore.store(AbstractFilesystemStore.java:150)
at
org.apache.excalibur.store.impl.MRUMemoryStore.dispose(MRUMemoryStore.java:129)
at
org.apache.avalon.excalibur.component.DefaultComponentFactory.decommission(DefaultComponentFactory.java:320)
RE: Performance leak
Posted by Stephan Michels <st...@apache.org>.
On Thu, 6 Mar 2003, Carsten Ziegeler wrote:
>
> Stephan Michels wrote:
> >
> > I do some profiling today, and wondering why the XSLT transformer
> > burns performance like hell in setup(). After some debugging I noticed
> > that the XSLT transformer create a complete new template/handler for
> > each call of setup(), that because the parameter 'use-store' in
> > cocoon.xconf is set to 'false' per default.
> >
> > After setting this parameter to 'true', the performance increased
> > dramatically.
> >
> > So, is there a reason, why this parameter is set to 'false'? When
> > not, I would change it to 'true'.
> >
> There was a reason: a memory leak :) So you can change from performance
> to memory leak. This leak was at least in last Xalan versions, I never
> tested if it is in the latest, so perhaps this is fixed now.
> If not, you can run into serious memory problems.
Doh! ;-) I'll make some longtime stability tests, to verify this. The
memory leak should also occur if you use the store, except if you forget
to remove the entries. At the moment, the concept of the XSLTC(and other)
doesn't make any sense, because you compile all used stylesheets for
every request :-/
I'll try to check this, Stephan Michels.
RE: Performance leak
Posted by Carsten Ziegeler <cz...@s-und-n.de>.
Stephan Michels wrote:
>
> I do some profiling today, and wondering why the XSLT transformer
> burns performance like hell in setup(). After some debugging I noticed
> that the XSLT transformer create a complete new template/handler for
> each call of setup(), that because the parameter 'use-store' in
> cocoon.xconf is set to 'false' per default.
>
> After setting this parameter to 'true', the performance increased
> dramatically.
>
> So, is there a reason, why this parameter is set to 'false'? When
> not, I would change it to 'true'.
>
There was a reason: a memory leak :) So you can change from performance
to memory leak. This leak was at least in last Xalan versions, I never
tested if it is in the latest, so perhaps this is fixed now.
If not, you can run into serious memory problems.
Carsten