You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lenya.apache.org by Josias Thoeny <jo...@wyona.com> on 2005/04/08 11:07:30 UTC
Re: DO NOT REPLY [Bug 32840] - replace sitetree script
with one based on XMLHTTP
<snip/>
> Before, the sitetree was generated from the sitetree.xml file.
> But now, the SiteTreeGenerator uses the SiteTree object, currently
> a DefaultSiteTree, which holds the sitetree in a DOM.
>
> Generating the sitetree from the DOM means heavy load on the DOM
> itself and since the getLabels() method of SiteTreeNodeImpl access
> the DOM in an asynchronous manner, it is most likely that the DOM
> get corrupted (bug 33908).
Concerning the performance, something else just came to my mind:
It might make sense for the SiteTreeGenerator to implement the
CacheableProcessingComponent interface. That way, its output could be
cached by cocoon, which should increase the performance.
The problem is how to know whether a cached result is still valid (i.e.
how to implement the SourceValidity).
Anyway, some profiling might be useful to see if this is really an
issue.
Josias
>
> The proposed interface changes in bug 34348 adds a reference to
> the sitetree in SiteTreeNodeImpl which can be used to synchronize the
> access to the DOM. I will update the sources in that way and
> re-attach it to the proposal bug.
>
> Regards,
> Felix
>
>
> >
> > Josias
> >
> >
> >>Here is excerpt from the stacktrce:
> >>
> >>java.lang.NullPointerException
> >> at
> >>org.apache.lenya.cms.publication.SiteTreeNodeImpl.getLabels(SiteTreeNodeImpl.java:146)
> >> at
> >>org.apache.lenya.cms.cocoon.generation.SiteTreeGenerator.generateNodes(SiteTreeGenerator.java:146)
> >> at
> >>org.apache.lenya.cms.cocoon.generation.SiteTreeGenerator.generateNodes(SiteTreeGenerator.java:151)
> >> at
> >>org.apache.lenya.cms.cocoon.generation.SiteTreeGenerator.generateNodes(SiteTreeGenerator.java:151)
> >> at
> >>org.apache.lenya.cms.cocoon.generation.SiteTreeGenerator.generateNodes(SiteTreeGenerator.java:151)
> >> at
> >>org.apache.lenya.cms.cocoon.generation.SiteTreeGenerator.generateSiteTree(SiteTreeGenerator.java:128)
> >> at
> >>org.apache.lenya.cms.cocoon.generation.SiteTreeGenerator.generate(SiteTreeGenerator.java:111)
> >> at
> >>org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.processXMLPipeline(AbstractProcessingPipeline.java:530)
> >> at
> >>org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline.processXMLPipeline(AbstractCachingProcessingPipeline.java:172)
> >> at
> >>org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.process(AbstractProcessingPipeline.java:701)
> >> at
> >>org.apache.cocoon.components.source.impl.SitemapSource.toSAX(SitemapSource.java:415)
> >> at
> >>org.apache.cocoon.components.source.SourceUtil.parse(SourceUtil.java:264)
> >> at
> >>org.apache.cocoon.generation.FileGenerator.generate(FileGenerator.java:117)
> >> at
> >>org.apache.cocoon.components.pipeline.impl.AbstractCachingProcessingPipeline.processXMLPipeline(AbstractCachingProcessingPipeline.java:224)
> >> at
> >>org.apache.cocoon.components.pipeline.AbstractProcessingPipeline.process(AbstractProcessingPipeline.java:701)
> >> at
> >>org.apache.cocoon.components.source.impl.SitemapSource.toSAX(SitemapSource.java:415)
> >> at
> >>org.apache.cocoon.components.source.SourceUtil.parse(SourceUtil.java:264)
> >> at
> >>org.apache.cocoon.sitemap.ContentAggregator.generate(ContentAggregator.java:125)
> >>
> >>[ ... ]
> >>
> >>Jann
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
> For additional commands, e-mail: dev-help@lenya.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: DO NOT REPLY [Bug 32840] - replace sitetree script
withone based on XMLHTTP
Posted by jo...@wyona.com.
> Josias Thoeny wrote:
>> <snip/>
>>
>>>Before, the sitetree was generated from the sitetree.xml file.
>>>But now, the SiteTreeGenerator uses the SiteTree object, currently
>>>a DefaultSiteTree, which holds the sitetree in a DOM.
>>>
>>>Generating the sitetree from the DOM means heavy load on the DOM
>>>itself and since the getLabels() method of SiteTreeNodeImpl access
>>>the DOM in an asynchronous manner, it is most likely that the DOM
>>>get corrupted (bug 33908).
>>
>>
>> Concerning the performance, something else just came to my mind:
>> It might make sense for the SiteTreeGenerator to implement the
>> CacheableProcessingComponent interface. That way, its output could be
>> cached by cocoon, which should increase the performance.
>> The problem is how to know whether a cached result is still valid (i.e.
>> how to implement the SourceValidity).
>> Anyway, some profiling might be useful to see if this is really an
>> issue.
>
> Wow, you're already quite into the internals ...
> I'm looking forward to your help in the design of 1.4 :)
I certainly would like to help with 1.4, it's just lack of time...
I guess that's everyone's problem.
Currently I'm still working on the sitetree script replacement for 1.4,
problably that will be finished soon.
Josias
> -- Andreas
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
> For additional commands, e-mail: dev-help@lenya.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: DO NOT REPLY [Bug 32840] - replace sitetree script with
one based on XMLHTTP
Posted by Andreas Hartmann <an...@apache.org>.
Josias Thoeny wrote:
> <snip/>
>
>>Before, the sitetree was generated from the sitetree.xml file.
>>But now, the SiteTreeGenerator uses the SiteTree object, currently
>>a DefaultSiteTree, which holds the sitetree in a DOM.
>>
>>Generating the sitetree from the DOM means heavy load on the DOM
>>itself and since the getLabels() method of SiteTreeNodeImpl access
>>the DOM in an asynchronous manner, it is most likely that the DOM
>>get corrupted (bug 33908).
>
>
> Concerning the performance, something else just came to my mind:
> It might make sense for the SiteTreeGenerator to implement the
> CacheableProcessingComponent interface. That way, its output could be
> cached by cocoon, which should increase the performance.
> The problem is how to know whether a cached result is still valid (i.e.
> how to implement the SourceValidity).
> Anyway, some profiling might be useful to see if this is really an
> issue.
Wow, you're already quite into the internals ...
I'm looking forward to your help in the design of 1.4 :)
-- Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: DO NOT REPLY [Bug 32840] - replace sitetree script with one
based on XMLHTTP
Posted by Jann Forrer <ja...@id.unizh.ch>.
On Fri, 8 Apr 2005, Josias Thoeny wrote:
> <snip/>
>
> > Before, the sitetree was generated from the sitetree.xml file.
> > But now, the SiteTreeGenerator uses the SiteTree object, currently
> > a DefaultSiteTree, which holds the sitetree in a DOM.
> >
> > Generating the sitetree from the DOM means heavy load on the DOM
> > itself and since the getLabels() method of SiteTreeNodeImpl access
> > the DOM in an asynchronous manner, it is most likely that the DOM
> > get corrupted (bug 33908).
>
I did apply the changes proposed by Felix in bug 34348 and I get rid of
the error messages and the disorganization of the sitetree due to above
mentioned problems :-) (which did occur when i put heavy load on a lenya
live site). Now I got zero error messages and the sitetree keeps in good
order :-)
> Concerning the performance, something else just came to my mind:
> It might make sense for the SiteTreeGenerator to implement the
> CacheableProcessingComponent interface. That way, its output could be
> cached by cocoon, which should increase the performance.
> The problem is how to know whether a cached result is still valid (i.e.
> how to implement the SourceValidity).
> Anyway, some profiling might be useful to see if this is really an
> issue.
>
As alread mentioned I did some load testing using JMeter. The Results are
probably of interest. I test a lenya live site access it directy via
servlet engine (tomcat) and the same site static exported (wget) access it
via apache.
My Computer:
linux : Linux version 2.6.8 (SuSE 9.2)
model name : Intel(R) Pentium(R) 4 CPU 3.20GHz
cache size : 1024 KB
Testplan:
50 concurrent user access 10 times 2 pages (1000 Request)
Results (summarized):
Tomcat: 1.5 request / second
Apache: 100 request / sedond
There is certainly room to enhance the lenya/tomcat performance. Some
possibilites have already been discussed on this lis e.g.
mod_lenya, simple caching mechanism (as described by Michi Wechner some
times ago), using apache to access images ......
Jann
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: DO NOT REPLY [Bug 32840] - replace sitetree script with one
based on XMLHTTP
Posted by Felix Röthenbacher <fe...@wyona.com>.
Josias Thoeny wrote:
...
>>
>>It's a good idea anyway! ;-)
>>It really makes sense that the SiteTreeGenerator implements the
>>CacheableProcessingComponent. The SourceValidity object returned
>>by the interface will ask the sitetree for the last modification
>>timestamp. This calls for an interface the sitetree implementation
>>implements to return the last modification timestamp to the
>>validity object.
>>
>>WDYT??
>
>
> To me this approach seems quite reasonable.
> IIUC a sitetree implementation which does not want to be cached just
> would not implement the "LastModified" interface (maybe the interface
> should be called "CacheableSiteTree" or something like that?).
I'd prefer the more generic name LastModified because it's not
bound to sitetrees only.
>
> I think the following part of your patch should be "if _not_
> instanceof":
>
> + if (sitetree instanceof LastModified) {
> + return null;
> + } else {
> + return new TimeStampValidity(((LastModified)sitetree).getLastModified());
> + }
>
> Or am I wrong?
You are right! Thanks for spotting.
>
> Josias
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: DO NOT REPLY [Bug 32840] - replace sitetree
script with one based on XMLHTTP
Posted by Josias Thoeny <jo...@wyona.com>.
On Fri, 2005-04-08 at 12:16 +0200, Felix Röthenbacher wrote:
> Josias Thoeny wrote:
> > <snip/>
> >
> >>Before, the sitetree was generated from the sitetree.xml file.
> >>But now, the SiteTreeGenerator uses the SiteTree object, currently
> >>a DefaultSiteTree, which holds the sitetree in a DOM.
> >>
> >>Generating the sitetree from the DOM means heavy load on the DOM
> >>itself and since the getLabels() method of SiteTreeNodeImpl access
> >>the DOM in an asynchronous manner, it is most likely that the DOM
> >>get corrupted (bug 33908).
> >
> >
> > Concerning the performance, something else just came to my mind:
> > It might make sense for the SiteTreeGenerator to implement the
> > CacheableProcessingComponent interface. That way, its output could be
> > cached by cocoon, which should increase the performance.
> > The problem is how to know whether a cached result is still valid (i.e.
> > how to implement the SourceValidity).
> > Anyway, some profiling might be useful to see if this is really an
> > issue.
> >
>
> It's a good idea anyway! ;-)
> It really makes sense that the SiteTreeGenerator implements the
> CacheableProcessingComponent. The SourceValidity object returned
> by the interface will ask the sitetree for the last modification
> timestamp. This calls for an interface the sitetree implementation
> implements to return the last modification timestamp to the
> validity object.
>
> WDYT??
To me this approach seems quite reasonable.
IIUC a sitetree implementation which does not want to be cached just
would not implement the "LastModified" interface (maybe the interface
should be called "CacheableSiteTree" or something like that?).
I think the following part of your patch should be "if _not_
instanceof":
+ if (sitetree instanceof LastModified) {
+ return null;
+ } else {
+ return new TimeStampValidity(((LastModified)sitetree).getLastModified());
+ }
Or am I wrong?
Josias
>
> - Felix
>
> > Josias
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
> For additional commands, e-mail: dev-help@lenya.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: DO NOT REPLY [Bug 32840] - replace sitetree script with one
based on XMLHTTP
Posted by Felix Röthenbacher <fe...@wyona.com>.
Josias Thoeny wrote:
> <snip/>
>
>>Before, the sitetree was generated from the sitetree.xml file.
>>But now, the SiteTreeGenerator uses the SiteTree object, currently
>>a DefaultSiteTree, which holds the sitetree in a DOM.
>>
>>Generating the sitetree from the DOM means heavy load on the DOM
>>itself and since the getLabels() method of SiteTreeNodeImpl access
>>the DOM in an asynchronous manner, it is most likely that the DOM
>>get corrupted (bug 33908).
>
>
> Concerning the performance, something else just came to my mind:
> It might make sense for the SiteTreeGenerator to implement the
> CacheableProcessingComponent interface. That way, its output could be
> cached by cocoon, which should increase the performance.
> The problem is how to know whether a cached result is still valid (i.e.
> how to implement the SourceValidity).
> Anyway, some profiling might be useful to see if this is really an
> issue.
>
It's a good idea anyway! ;-)
It really makes sense that the SiteTreeGenerator implements the
CacheableProcessingComponent. The SourceValidity object returned
by the interface will ask the sitetree for the last modification
timestamp. This calls for an interface the sitetree implementation
implements to return the last modification timestamp to the
validity object.
WDYT??
- Felix
> Josias
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org