You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lenya.apache.org by Dominique <op...@gmail.com> on 2007/05/08 16:34:40 UTC
Re: very large number of documents
as you may have noticed, I bundled my quite succesfull efforts to
speedup Lenya with large sitetrees at the following wiki:
http://wiki.apache.org/lenya/Large%20number%20of%20documents%20HOWTO
I put it under Tutorials/Performance.
Note that I left the sitetree.xml concept, did some tests and even
with 100 000 docs response times were acceptable (which is all I
need).
Any comments are welcome !
bye,
Dominique
On 27/04/07, Michael Ralston <mi...@ralston.id.au> wrote:
> On 4/26/07, Andreas Hartmann <an...@apache.org> wrote:
> > Dominique schrieb:
> > > Hi devs,
> > >
> > > I read the posts of someone who used an-EJB backend for the
> > > repository, but integrating
> > > such a solution seems a lot of work and needs JBoss.
> > > I was thinking of replacing the one big "sitetree.xml" with smaller
> > > ones, one at each subfolder level. This would involve reimpmenting the
> > > Sitetree interface,
> >
> > A site manager implementation that uses an RDBMS (e.g. via Hibernate)
> > and maybe some caching might also be a reasonable approach.
> > Basically you just need to store the node properties and
> > parent-child relations.
> >
>
> At one stage I attempted to get my EJB backend to use a relational
> model to render the sitetree. It didn't work very well, my biggest
> problem was trying to delete a node from the tree. As the UnitOfWork
> is the only thing which ultimately writes to the database, I had to
> make each SiteTreeNode implement transactionable.
>
> The other problem I found, was that requests for sitetree.xml just ask
> for the contents of the entire file. My ejb sitetree had to output
> that xml, I tried various methods of caching that but it turned out to
> be very inefficient.
>
> I'm now just storing the sitetree in xml format the same as the normal
> lenya repository. If anyone has some suggestions of how to change the
> cocoon pipelines which handle requests for the sitetree, so that it
> can request a subset of the tree, I'd be interested in trying to
> implement that.
>
> Michael Ralston
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
> For additional commands, e-mail: dev-help@lenya.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: very large number of documents
Posted by Jörn Nettingsmeier <ne...@apache.org>.
Dominique wrote:
> as you may have noticed, I bundled my quite succesfull efforts to
> speedup Lenya with large sitetrees at the following wiki:
>
>
> http://wiki.apache.org/lenya/Large%20number%20of%20documents%20HOWTO
>
>
> I put it under Tutorials/Performance.
> Note that I left the sitetree.xml concept, did some tests and even
> with 100 000 docs response times were acceptable (which is all I
> need).
>
> Any comments are welcome !
thanks for this valuable contribution!
--
Jörn Nettingsmeier
Kurt is up in heaven now.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: very large number of documents
Posted by Dominique <op...@gmail.com>.
I did this for 1.2, since I'm only familiar with that (and just
updated the Wiki to make that clear). If I find the time I may look at
the 1.4 and maybe submit some patches.
Functionality changes:
-The default "menu.xsl" in Lenya 1.2 shows, beside the children of the
current nodee, also all "children nodes" of the brothers/sisters of
the current node. With the selective-sitetree generator this will not
happen (in its current implementation). Yet I found this default
behaviour counter-intuitive and it was one of the first things I
removed even before my selective-sitetree. ( removed 1
<xsl:apply-templates select="nav:node"/> in menu.xsl )
-The delete, archive .... usecases will not check for links that will
not work anymore
On 08/05/07, Michael Wechner <mi...@wyona.com> wrote:
> Dominique wrote:
>
> > as you may have noticed, I bundled my quite succesfull efforts to
> > speedup Lenya with large sitetrees at the following wiki:
> >
> >
> > http://wiki.apache.org/lenya/Large%20number%20of%20documents%20HOWTO
> >
> >
> > I put it under Tutorials/Performance.
>
>
> this sounds great :-) Did you do this for 1.2 or 1.4?
>
> Would it make sense to apply this as a patch or does it leave some other
> functionality out?
>
> Cheers
>
> Michael
>
> > Note that I left the sitetree.xml concept, did some tests and even
> > with 100 000 docs response times were acceptable (which is all I
> > need).
> >
> > Any comments are welcome !
> >
> > bye,
> >
> > Dominique
> >
> > On 27/04/07, Michael Ralston <mi...@ralston.id.au> wrote:
> >
> >> On 4/26/07, Andreas Hartmann <an...@apache.org> wrote:
> >> > Dominique schrieb:
> >> > > Hi devs,
> >> > >
> >> > > I read the posts of someone who used an-EJB backend for the
> >> > > repository, but integrating
> >> > > such a solution seems a lot of work and needs JBoss.
> >> > > I was thinking of replacing the one big "sitetree.xml" with smaller
> >> > > ones, one at each subfolder level. This would involve
> >> reimpmenting the
> >> > > Sitetree interface,
> >> >
> >> > A site manager implementation that uses an RDBMS (e.g. via Hibernate)
> >> > and maybe some caching might also be a reasonable approach.
> >> > Basically you just need to store the node properties and
> >> > parent-child relations.
> >> >
> >>
> >> At one stage I attempted to get my EJB backend to use a relational
> >> model to render the sitetree. It didn't work very well, my biggest
> >> problem was trying to delete a node from the tree. As the UnitOfWork
> >> is the only thing which ultimately writes to the database, I had to
> >> make each SiteTreeNode implement transactionable.
> >>
> >> The other problem I found, was that requests for sitetree.xml just ask
> >> for the contents of the entire file. My ejb sitetree had to output
> >> that xml, I tried various methods of caching that but it turned out to
> >> be very inefficient.
> >>
> >> I'm now just storing the sitetree in xml format the same as the normal
> >> lenya repository. If anyone has some suggestions of how to change the
> >> cocoon pipelines which handle requests for the sitetree, so that it
> >> can request a subset of the tree, I'd be interested in trying to
> >> implement that.
> >>
> >> Michael Ralston
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
> >> For additional commands, e-mail: dev-help@lenya.apache.org
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
> > For additional commands, e-mail: dev-help@lenya.apache.org
> >
> >
>
>
> --
> Michael Wechner
> Wyona - Open Source Content Management - Apache Lenya
> http://www.wyona.com http://lenya.apache.org
> michael.wechner@wyona.com michi@apache.org
> +41 44 272 91 61
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
> For additional commands, e-mail: dev-help@lenya.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org
Re: very large number of documents
Posted by Michael Wechner <mi...@wyona.com>.
Dominique wrote:
> as you may have noticed, I bundled my quite succesfull efforts to
> speedup Lenya with large sitetrees at the following wiki:
>
>
> http://wiki.apache.org/lenya/Large%20number%20of%20documents%20HOWTO
>
>
> I put it under Tutorials/Performance.
this sounds great :-) Did you do this for 1.2 or 1.4?
Would it make sense to apply this as a patch or does it leave some other
functionality out?
Cheers
Michael
> Note that I left the sitetree.xml concept, did some tests and even
> with 100 000 docs response times were acceptable (which is all I
> need).
>
> Any comments are welcome !
>
> bye,
>
> Dominique
>
> On 27/04/07, Michael Ralston <mi...@ralston.id.au> wrote:
>
>> On 4/26/07, Andreas Hartmann <an...@apache.org> wrote:
>> > Dominique schrieb:
>> > > Hi devs,
>> > >
>> > > I read the posts of someone who used an-EJB backend for the
>> > > repository, but integrating
>> > > such a solution seems a lot of work and needs JBoss.
>> > > I was thinking of replacing the one big "sitetree.xml" with smaller
>> > > ones, one at each subfolder level. This would involve
>> reimpmenting the
>> > > Sitetree interface,
>> >
>> > A site manager implementation that uses an RDBMS (e.g. via Hibernate)
>> > and maybe some caching might also be a reasonable approach.
>> > Basically you just need to store the node properties and
>> > parent-child relations.
>> >
>>
>> At one stage I attempted to get my EJB backend to use a relational
>> model to render the sitetree. It didn't work very well, my biggest
>> problem was trying to delete a node from the tree. As the UnitOfWork
>> is the only thing which ultimately writes to the database, I had to
>> make each SiteTreeNode implement transactionable.
>>
>> The other problem I found, was that requests for sitetree.xml just ask
>> for the contents of the entire file. My ejb sitetree had to output
>> that xml, I tried various methods of caching that but it turned out to
>> be very inefficient.
>>
>> I'm now just storing the sitetree in xml format the same as the normal
>> lenya repository. If anyone has some suggestions of how to change the
>> cocoon pipelines which handle requests for the sitetree, so that it
>> can request a subset of the tree, I'd be interested in trying to
>> implement that.
>>
>> Michael Ralston
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
>> For additional commands, e-mail: dev-help@lenya.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
> For additional commands, e-mail: dev-help@lenya.apache.org
>
>
--
Michael Wechner
Wyona - Open Source Content Management - Apache Lenya
http://www.wyona.com http://lenya.apache.org
michael.wechner@wyona.com michi@apache.org
+41 44 272 91 61
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lenya.apache.org
For additional commands, e-mail: dev-help@lenya.apache.org