You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cocoon.apache.org by "Superbiji@gmail.com" <su...@gmail.com> on 2004/10/11 08:21:05 UTC
aggregate takes too much time
Hi all,
Aggregating xml in cocoon takes too much time.
I try to aggregate >10 xml files ( from filesystem: file:/// )
It takes about 4-7 seconds just to aggregate w/o transforming :(
Any other faster way to aggregate contents?
--
Indonesia Cocoon: idcocoon-subscribe @ yahoogroups.com
Superbiji @ gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: aggregate takes too much time
Posted by Emmanuel Charpentier <em...@free.fr>.
Ever used xpathdirectorygenerator?
http://cocoon.apache.org/2.1/userdocs/generators/xpathdirectory-generator.html
It could be another solution to pit against aggregates
manu
On Mon, Oct 11, 2004 at 02:25:34PM +0700, Superbiji@gmail.com wrote:
> no simple aggregation of several local xml files.
> and one transformation....
>
> I use profiler, the transformation takes 412 ms.
> serialization takes 20 ms.
>
>
> On Mon, 11 Oct 2004 09:06:14 +0200, Jan Hoskens <jh...@schaubroeck.be> wrote:
> > Are you using very large documents? Do you have other pipeline
> > structures in between the aggregation or afterwards that may slow things
> > down? Why do you need to aggregate more than 10 documents at a time?
> >
> > (Not sure I can help, but more info may be usefull for other readers too ;-)
> >
> > Kind Regards,
> > Jan
> >
> > Superbiji@gmail.com wrote:
> >
> > >XInclude is faster, but after several requests it slows down (don't know why)
> >
> >
> > >CInclude is as slow as <map:aggregate>.
> > >
> > >On Mon, 11 Oct 2004 08:30:01 +0200, Jan Hoskens <jh...@schaubroeck.be> wrote:
> > >
> > >
> > >>Have you tried different options? Besides the sitemap aggregation there
> > >>are also some include transformators available: the XInclude and
> > >>CInclude may do the same as the aggregation.
> > >>
> > >>Kind regards,
> > >>Jan
> > >>
> > >>Superbiji@gmail.com wrote:
> > >>
> > >>
> > >>
> > >>>Hi all,
> > >>>
> > >>>Aggregating xml in cocoon takes too much time.
> > >>>
> > >>>I try to aggregate >10 xml files ( from filesystem: file:/// )
> > >>>It takes about 4-7 seconds just to aggregate w/o transforming :(
> > >>>
> > >>>Any other faster way to aggregate contents?
> > >>>
> >
>
>
> --
> Indonesia Cocoon: idcocoon-subscribe @ yahoogroups.com
> Superbiji @ gmail.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
> For additional commands, e-mail: users-help@cocoon.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: aggregate takes too much time
Posted by "Christoph Gaffga (triplemind.com)" <cg...@triplemind.com>.
I didn't wanted to send a patch for that. The code is poorly documented nor
tested under many circumtances, old license, etc. etc. It was just a
workaround for a problem we had.
But we use it in production environment with about 350.000 pages a day. So,
for my usecase it's stable. And if you suggest to send it as a patch, I do.
Perheaps somebody finishes my work and we'll have it in the next version of
cocoon.
In addition to my ParallelContentAggregator class I have a version called
TonyContentAggregator. I can't remember if Tony did this, it was inspired by
Tony or whatelse. I also attached thisone as patch.
I have posted it as an enhancement to the bugzilla.
So, see attachements to the bugreport
http://nagoya.apache.org/bugzilla/show_bug.cgi?id=31646
for the javacode.
Maybe somebody put this in the scratchpad.
regards
Christoph
----- Original Message -----
From: "Ralph Goers" <Ra...@dslextreme.com>
To: <us...@cocoon.apache.org>
Sent: Monday, October 11, 2004 4:37 PM
Subject: Re: aggregate takes too much time
> Why not submit this as a patch? Then we can all see it.
>
> Christoph Gaffga wrote:
>
> >we also had a problem with slow aggregates. We solved it by changing the
> >Aggregator so it works in a multithraded mode.
> >
> >I don't know if it increases the perfomcance in your case because you are
> >only aggregating simple xml files from the filesystem. But in our case
(the
> >aggragated parts are not static) it increases perfomance dramaticly.
> >
> >If you interested in it, I can send you our javacode for the
> >ParallelContentAggregator.
> >
> >regards
> >Christoph
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
> For additional commands, e-mail: users-help@cocoon.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: aggregate takes too much time
Posted by Ralph Goers <Ra...@dslextreme.com>.
Why not submit this as a patch? Then we can all see it.
Christoph Gaffga wrote:
>we also had a problem with slow aggregates. We solved it by changing the
>Aggregator so it works in a multithraded mode.
>
>I don't know if it increases the perfomcance in your case because you are
>only aggregating simple xml files from the filesystem. But in our case (the
>aggragated parts are not static) it increases perfomance dramaticly.
>
>If you interested in it, I can send you our javacode for the
>ParallelContentAggregator.
>
>regards
>Christoph
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: aggregate takes too much time
Posted by Christoph Gaffga <cg...@triplemind.com>.
we also had a problem with slow aggregates. We solved it by changing the
Aggregator so it works in a multithraded mode.
I don't know if it increases the perfomcance in your case because you are
only aggregating simple xml files from the filesystem. But in our case (the
aggragated parts are not static) it increases perfomance dramaticly.
If you interested in it, I can send you our javacode for the
ParallelContentAggregator.
regards
Christoph
"Superbiji@gmail.com" <su...@gmail.com> schrieb im Newsbeitrag
news:cc008e72041011002543a8b9ca@mail.gmail.com...
> no simple aggregation of several local xml files.
> and one transformation....
>
> I use profiler, the transformation takes 412 ms.
> serialization takes 20 ms.
>
>
> On Mon, 11 Oct 2004 09:06:14 +0200, Jan Hoskens <jh...@schaubroeck.be> wrote:
> > Are you using very large documents? Do you have other pipeline
> > structures in between the aggregation or afterwards that may slow things
> > down? Why do you need to aggregate more than 10 documents at a time?
> >
> > (Not sure I can help, but more info may be usefull for other readers too
;-)
> >
> > Kind Regards,
> > Jan
> >
> > Superbiji@gmail.com wrote:
> >
> > >XInclude is faster, but after several requests it slows down (don't
know why)
> >
> >
> > >CInclude is as slow as <map:aggregate>.
> > >
> > >On Mon, 11 Oct 2004 08:30:01 +0200, Jan Hoskens <jh...@schaubroeck.be>
wrote:
> > >
> > >
> > >>Have you tried different options? Besides the sitemap aggregation
there
> > >>are also some include transformators available: the XInclude and
> > >>CInclude may do the same as the aggregation.
> > >>
> > >>Kind regards,
> > >>Jan
> > >>
> > >>Superbiji@gmail.com wrote:
> > >>
> > >>
> > >>
> > >>>Hi all,
> > >>>
> > >>>Aggregating xml in cocoon takes too much time.
> > >>>
> > >>>I try to aggregate >10 xml files ( from filesystem: file:/// )
> > >>>It takes about 4-7 seconds just to aggregate w/o transforming :(
> > >>>
> > >>>Any other faster way to aggregate contents?
> > >>>
> >
>
>
> --
> Indonesia Cocoon: idcocoon-subscribe @ yahoogroups.com
> Superbiji @ gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: aggregate takes too much time
Posted by "Superbiji@gmail.com" <su...@gmail.com>.
no simple aggregation of several local xml files.
and one transformation....
I use profiler, the transformation takes 412 ms.
serialization takes 20 ms.
On Mon, 11 Oct 2004 09:06:14 +0200, Jan Hoskens <jh...@schaubroeck.be> wrote:
> Are you using very large documents? Do you have other pipeline
> structures in between the aggregation or afterwards that may slow things
> down? Why do you need to aggregate more than 10 documents at a time?
>
> (Not sure I can help, but more info may be usefull for other readers too ;-)
>
> Kind Regards,
> Jan
>
> Superbiji@gmail.com wrote:
>
> >XInclude is faster, but after several requests it slows down (don't know why)
>
>
> >CInclude is as slow as <map:aggregate>.
> >
> >On Mon, 11 Oct 2004 08:30:01 +0200, Jan Hoskens <jh...@schaubroeck.be> wrote:
> >
> >
> >>Have you tried different options? Besides the sitemap aggregation there
> >>are also some include transformators available: the XInclude and
> >>CInclude may do the same as the aggregation.
> >>
> >>Kind regards,
> >>Jan
> >>
> >>Superbiji@gmail.com wrote:
> >>
> >>
> >>
> >>>Hi all,
> >>>
> >>>Aggregating xml in cocoon takes too much time.
> >>>
> >>>I try to aggregate >10 xml files ( from filesystem: file:/// )
> >>>It takes about 4-7 seconds just to aggregate w/o transforming :(
> >>>
> >>>Any other faster way to aggregate contents?
> >>>
>
--
Indonesia Cocoon: idcocoon-subscribe @ yahoogroups.com
Superbiji @ gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: aggregate takes too much time
Posted by Jan Hoskens <jh...@schaubroeck.be>.
Are you using very large documents? Do you have other pipeline
structures in between the aggregation or afterwards that may slow things
down? Why do you need to aggregate more than 10 documents at a time?
(Not sure I can help, but more info may be usefull for other readers too ;-)
Kind Regards,
Jan
Superbiji@gmail.com wrote:
>XInclude is faster, but after several requests it slows down (don't know why)
>CInclude is as slow as <map:aggregate>.
>
>On Mon, 11 Oct 2004 08:30:01 +0200, Jan Hoskens <jh...@schaubroeck.be> wrote:
>
>
>>Have you tried different options? Besides the sitemap aggregation there
>>are also some include transformators available: the XInclude and
>>CInclude may do the same as the aggregation.
>>
>>Kind regards,
>>Jan
>>
>>Superbiji@gmail.com wrote:
>>
>>
>>
>>>Hi all,
>>>
>>>Aggregating xml in cocoon takes too much time.
>>>
>>>I try to aggregate >10 xml files ( from filesystem: file:/// )
>>>It takes about 4-7 seconds just to aggregate w/o transforming :(
>>>
>>>Any other faster way to aggregate contents?
>>>
>>>
>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: aggregate takes too much time
Posted by "Superbiji@gmail.com" <su...@gmail.com>.
XInclude is faster, but after several requests it slows down (don't know why)
CInclude is as slow as <map:aggregate>.
On Mon, 11 Oct 2004 08:30:01 +0200, Jan Hoskens <jh...@schaubroeck.be> wrote:
> Have you tried different options? Besides the sitemap aggregation there
> are also some include transformators available: the XInclude and
> CInclude may do the same as the aggregation.
>
> Kind regards,
> Jan
>
> Superbiji@gmail.com wrote:
>
> >Hi all,
> >
> >Aggregating xml in cocoon takes too much time.
> >
> >I try to aggregate >10 xml files ( from filesystem: file:/// )
> >It takes about 4-7 seconds just to aggregate w/o transforming :(
> >
> >Any other faster way to aggregate contents?
> >
--
Indonesia Cocoon: idcocoon-subscribe @ yahoogroups.com
Superbiji @ gmail.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org
Re: aggregate takes too much time
Posted by Jan Hoskens <jh...@schaubroeck.be>.
Have you tried different options? Besides the sitemap aggregation there
are also some include transformators available: the XInclude and
CInclude may do the same as the aggregation.
Kind regards,
Jan
Superbiji@gmail.com wrote:
>Hi all,
>
>Aggregating xml in cocoon takes too much time.
>
>I try to aggregate >10 xml files ( from filesystem: file:/// )
>It takes about 4-7 seconds just to aggregate w/o transforming :(
>
>Any other faster way to aggregate contents?
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org