You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Carsten Ziegeler <cz...@sundn.de> on 2001/01/17 13:02:41 UTC
AW: Status of XMLCompiler - Please vote...
Hi,
sorry, but I don't give up on this easy, so again:
The ozone projects uses the XMLCompiler, DB Prism will use it,
we will use it in our sunShine product, and I believe, there are
some others, which didn't came up with a response on this list,
who are using the XMLCompiler as well.
So again my question, why not integration it into C2?
Or we can make an proposal to Avalon or Xerces to integrate it.
I think, it doesn't make sense, that a lot of products have
their own version of the XMLCompiler. You can call it a
'maintainance nightmare'. And the further development of the
XMLCompiler is nearly impossible as it is spreaded around the
world.
Integrating the XMLCompiler doesn't hurt at all as no existing
component is affect, no interface has to change - simply nothing,
just adding classes.
But it would add a lot of value to C2.
So my vote for integrating XMLCompiler is
+100000
And YOU?
(OK, I know, I can't start a vote - but it's worth trying...)
Carsten
Re: AW: Status of XMLCompiler - Please vote...
Posted by Giacomo Pati <gi...@apache.org>.
Carsten Ziegeler wrote:
> Hi,
>
> sorry, but I don't give up on this easy, so again:
>
> The ozone projects uses the XMLCompiler, DB Prism will use it,
> we will use it in our sunShine product, and I believe, there are
> some others, which didn't came up with a response on this list,
> who are using the XMLCompiler as well.
>
>
> So again my question, why not integration it into C2?
> Or we can make an proposal to Avalon or Xerces to integrate it.
>
> I think, it doesn't make sense, that a lot of products have
> their own version of the XMLCompiler. You can call it a
> 'maintainance nightmare'. And the further development of the
> XMLCompiler is nearly impossible as it is spreaded around the
> world.
>
> Integrating the XMLCompiler doesn't hurt at all as no existing
> component is affect, no interface has to change - simply nothing,
> just adding classes.
> But it would add a lot of value to C2.
>
> So my vote for integrating XMLCompiler is
> +100000
Well, we can vote about it but this doesn't change anything :)
It would be better if you can come over with a proposal how to use and
integrate a XMLCompiler into the C2 architecture
Giacomo
AW: Status of XMLCompiler - Please vote...
Posted by Carsten Ziegeler <cz...@sundn.de>.
> Paul Russell wrote:
> Yep, agreed, although shouldn't the FileGenerator caching be done by the
> normal caching system? Rather than tying in something at a lower level?
This is not an easy question! Why not having both? Other components (XalanTransformer)
do it already and the low level implementation would speed up things, especially
in cases where the content is highly dynamic and incorporated by transformers.
In thoses cases would the general caching algorithm fail, but the low level
ones would speed this up.
> I'd rather see XMLCompiler integrated by having it as *one of a number*
> of potential XML, um, 'transcoders' (thanks batik) usable within the
> caching infrastructure. We've already had one offer of help with the
> caching architecture -- would you be interested in teaming up on this?
Yes of course, I would. I am still waiting for a proposal which is implementable
as I am currently stuck with caching of actions and selectors.
Carsten
Re: AW: Status of XMLCompiler - Please vote...
Posted by Paul Russell <pa...@luminas.co.uk>.
* Carsten Ziegeler (cziegeler@sundn.de) wrote :
> Believe me, I am familiar with XP....Of course, you are right: just adding
> classes which are never be used by the project itself, should be avoided.
Good good.
> OK, I didn't want to come up with this at this stage, but e.g. I could
> integrate the XMLCompiler into the FileGenerator for caching. So the
> FileGenerator can cache the parsed file using the XMLCompiler. This would
> speed up the FileGenerator (I think Stefano proposed that when he announced
> the XMLCompiler). This caching would be independent from the big caching
> algorithm C2 uses, like the caching of stylesheets in the XalanTransformer.
> In addition the big - yet-to-do - caching algorithm might need it, to cache
> partial responses. I don't mean caching at each stage in the pipeline!
> Sometimes it might be usefull to cache a response upto a specific point in the
> pipeline as all following stages are highly dynamic - but this is just a
> my thoguhts...
Yep, agreed, although shouldn't the FileGenerator caching be done by the
normal caching system? Rather than tying in something at a lower level?
I'd rather see XMLCompiler integrated by having it as *one of a number*
of potential XML, um, 'transcoders' (thanks batik) usable within the
caching infrastructure. We've already had one offer of help with the
caching architecture -- would you be interested in teaming up on this?
Paul
--
Paul Russell Email: paul@luminas.co.uk
Technical Director Tel: +44 (0)20 8553 6622
Luminas Internet Applications Fax: +44 (0)870 28 47489
This is not an official statement or order. Web: www.luminas.co.uk
AW: AW: Status of XMLCompiler - Please vote...
Posted by Carsten Ziegeler <cz...@sundn.de>.
> Berin Loritsch wrote:
>
>
> The Cocoon project calls them Generators. Let's keep the same name.
>
> Here is my spin on it: if the events will be called quicker by actually
> compiling a class and calling the methods like XSP, then we should
> determine by how much very easily.
>
> Take two files of roughly 100k each. The _only_ difference between them
> should be that one is treated as an XSP, and the other as an XML page.
> There should be no tranlators involved, and serialized as XML.
>
> NOTE: the XSP page may require the use of the <xsp:page> tags as the
> root.
>
> For an example of a small page like this, look at
> webapp/docs/samples/forms/add-department.xsp
>
> Access them each 11 times (throwing away the first time). Average the
> last ten times (the compile is the first access). If the average of
> the compiled sheet is significantly faster than the average of the
> uncompiled sheet, then it might be worth investigating this further.
> Otherwise, lets cut our losses because the RCT (Really Cool Technology)
> won't buy us enough of a performance edge.
>
> The next test is to create a Generator that reads the file into a string,
> and parses the string over and over--using the same test and test file.
> Then you will see what caching at the XML level will and will not do
> for you.
>
Attached are the performance results Stefano made when he startet the XMLCompiler.
> My take is that for static pages, we cache the final result. For
> dynamic pages we cache from the most static portion (Cocoon 2 really
> already does this for XSP--because the classloader caches the
> serverpages Generators).
I agree absolutely and for non XSP-pages the caching of the most static
portion *could* (not must) be done by the XMLCompiler.
Carsten
Re: AW: Status of XMLCompiler - Please vote...
Posted by Berin Loritsch <bl...@apache.org>.
Paul Russell wrote:
> * Sylvain Wallez (sylvain.wallez@anyware-tech.com) wrote :
>
>
>> BTW, if Ozone and DBPrism use XMLC, it could be better as part of Xerces
>
>> than Cocoon, even if it has no direct dependencies on Xerces.
>
>
>
> I reeeeally *really* don't like this. If it's not part of Xerces, it
>
> shouldn't be there, and if it's not part of cocoon, it shouldn't be
>
> there (at least not directly). Should it be a separate project in its
>
> own right? Perhaps there should be a project to develop a suite of these
>
> XML 'transcoders' (someone think of a better name for me? English never
>
> was my strong point. yes, I'm english ;)
The Cocoon project calls them Generators. Let's keep the same name.
Here is my spin on it: if the events will be called quicker by actually
compiling a class and calling the methods like XSP, then we should
determine by how much very easily.
Take two files of roughly 100k each. The _only_ difference between them
should be that one is treated as an XSP, and the other as an XML page.
There should be no tranlators involved, and serialized as XML.
NOTE: the XSP page may require the use of the <xsp:page> tags as the
root.
For an example of a small page like this, look at
webapp/docs/samples/forms/add-department.xsp
Access them each 11 times (throwing away the first time). Average the
last ten times (the compile is the first access). If the average of
the compiled sheet is significantly faster than the average of the
uncompiled sheet, then it might be worth investigating this further.
Otherwise, lets cut our losses because the RCT (Really Cool Technology)
won't buy us enough of a performance edge.
The next test is to create a Generator that reads the file into a string,
and parses the string over and over--using the same test and test file.
Then you will see what caching at the XML level will and will not do
for you.
My take is that for static pages, we cache the final result. For
dynamic pages we cache from the most static portion (Cocoon 2 really
already does this for XSP--because the classloader caches the
serverpages Generators).
Re: AW: Status of XMLCompiler - Please vote...
Posted by Paul Russell <pa...@luminas.co.uk>.
* Sylvain Wallez (sylvain.wallez@anyware-tech.com) wrote :
> BTW, if Ozone and DBPrism use XMLC, it could be better as part of Xerces
> than Cocoon, even if it has no direct dependencies on Xerces.
I reeeeally *really* don't like this. If it's not part of Xerces, it
shouldn't be there, and if it's not part of cocoon, it shouldn't be
there (at least not directly). Should it be a separate project in its
own right? Perhaps there should be a project to develop a suite of these
XML 'transcoders' (someone think of a better name for me? English never
was my strong point. yes, I'm english ;)
Paul.
--
Paul Russell Email: paul@luminas.co.uk
Technical Director Tel: +44 (0)20 8553 6622
Luminas Internet Applications Fax: +44 (0)870 28 47489
This is not an official statement or order. Web: www.luminas.co.uk
Re: AW: Status of XMLCompiler - Please vote...
Posted by Sylvain Wallez <sy...@anyware-tech.com>.
Paul Russell a écrit :
>
> * Carsten Ziegeler (cziegeler@sundn.de) wrote :
> > Integrating the XMLCompiler doesn't hurt at all as no existing
> > component is affect, no interface has to change - simply nothing,
> > just adding classes.
>
> Yeah, that's fine, but a very strong -1 for just adding classes
> because they 'might be useful'. If the cache needs them, then
> that's fine, if something else *core* needs them, then that's fine,
> but Cocoon2 is already 23,000 lines of code, let's not make it bigger
> than it has to be. For those familiar with XP, You Ain't Gonna Need it.
> For those who aren't <URL:http://www.extremeprogramming.org/> is *well*
> worth a read, it might just change the way you work.
>
I agree with you not to add unused stuff in the code base. However, we
may be facing a kind of chicken and egg problem : XMLCompiler will be
added when core components use it, but will core component use it if
it's not in the code base ?
XMLCompiler is an innovative idea (like many Stephano's idea) and people
may even not think of using it if it's not there. Also, C2 is still
unreleased. So why not integrate XMLC and remove it at release-time if
has been proven to be useless ?
BTW, if Ozone and DBPrism use XMLC, it could be better as part of Xerces
than Cocoon, even if it has no direct dependencies on Xerces.
> > (OK, I know, I can't start a vote - but it's worth trying...)
>
> Hahaha. We're open source. You can start what you like :)
>
> Paul
>
> --
> Paul Russell Email: paul@luminas.co.uk
> Technical Director Tel: +44 (0)20 8553 6622
> Luminas Internet Applications Fax: +44 (0)870 28 47489
> This is not an official statement or order. Web: www.luminas.co.uk
>
--
Sylvain Wallez
Anyware Technologies
AW: C2: Patch for FileGenerator and XMLCompiler
Posted by Carsten Ziegeler <cz...@sundn.de>.
> Um. Not wishing to be a pedant, but a 'Multiplexer' is something which
> squeezes multiple streams of communication into one, either by time or
> frequency division. May be better to call it 'Multicaster'.
>
Yes, I agree, you are right...perhaps I really should take some time to learn english....
Feel free to change the name.
Carsten
Re: C2: Patch for FileGenerator and XMLCompiler
Posted by Paul Russell <pa...@luminas.co.uk>.
* Carsten Ziegeler (cziegeler@sundn.de) wrote :
> The FileGenerator uses a new class XMLMultiplexer which can send SAXEvents
> to many ContentHandler. This class might also be useful for other purposes,
> e.g. caching...
Um. Not wishing to be a pedant, but a 'Multiplexer' is something which
squeezes multiple streams of communication into one, either by time or
frequency division. May be better to call it 'Multicaster'.
P.
--
Paul Russell Email: paul@luminas.co.uk
Technical Director Tel: +44 (0)20 8553 6622
Luminas Internet Applications Fax: +44 (0)870 28 47489
This is not an official statement or order. Web: www.luminas.co.uk
Re: C2: NEW Patch for FileGenerator and XMLCompiler
Posted by Paul Russell <pa...@luminas.co.uk>.
* Carsten Ziegeler (cziegeler@sundn.de) wrote :
> Hi,
>
> this is the same patch as some hours ago, but I renamed
> the XMLMultiplexer to XMLMulticaster (thanks to Paul)
> and fixed a bug in the XMLCompiler. Apologies for this.
Thanks Carsten,
I'll try and apply it a bit later, but I don't have any official Cocoon
time until this evening (UTC)..
Paul.
--
Paul Russell Email: paul@luminas.co.uk
Technical Director Tel: +44 (0)20 8553 6622
Luminas Internet Applications Fax: +44 (0)870 28 47489
This is not an official statement or order. Web: www.luminas.co.uk
C2: NEW Patch for FileGenerator and XMLCompiler
Posted by Carsten Ziegeler <cz...@sundn.de>.
Hi,
this is the same patch as some hours ago, but I renamed
the XMLMultiplexer to XMLMulticaster (thanks to Paul)
and fixed a bug in the XMLCompiler. Apologies for this.
Carsten
C2: Patch for FileGenerator and XMLCompiler
Posted by Carsten Ziegeler <cz...@sundn.de>.
Hi,
as promised: here is the second patch.
Included is the original XMLCompiler from Stefano (xmlcompiler.zip)
and a patch for the FileGenerator to use the XMLCompiler (diff.zip).
The FileGenerator stores (if configured to do so) each parsed
local XML file and retrieves it the next time it is requested
from the store.
The file modification date is used to invalidate the stored
data.
Configuration:
The generator can have a default configuration:
<map:generator ....>
<use-store map:value="true"/>
</map:generator>
The default is "false".
Each pipeline can override this behaviour:
<map:generate type="file" src="...">
<parameter name="use-store" value="false"/>
</map:generate>
I did some very quick tests for performance. The whole pipeline processing
increased with the processor depending on the load of the server between
10% and 30%. As the parsing of XML is only one step in the pipeline
the performance win is higher than that.
The FileGenerator uses a new class XMLMultiplexer which can send SAXEvents
to many ContentHandler. This class might also be useful for other purposes,
e.g. caching...
Carsten
C2: Patch for XalanTransformer
Posted by Carsten Ziegeler <cz...@sundn.de>.
Hi,
attached is the first promised patch for using the store.
The XalanTransformer now uses the store and not a
static variable to store the stylesheets.
Carsten
AW: Status of XMLCompiler - Please vote...
Posted by Carsten Ziegeler <cz...@sundn.de>.
> Giacomo Pati wrote:
>
> Don't mix cache and store. Cache is the thing we want to speed up
> processing
> almost transparent to each component. The FileGenerator (when it uses the
> XMLCompiler) can use the general component Store (as the
> Xalantransformer is
> doing) to store compiled xml files into. Integrating the
> XMLCompiler into the
> Filegenerator should be a snap, isn't it?
>
Yeah, should be - I will doing it today or tomorrow and send the patch asap.
I will also fix the XalanTransformer as it is currently not using the store
but a static variable to hold the stylesheets.
Carsten
Re: AW: Status of XMLCompiler - Please vote...
Posted by Giacomo Pati <gi...@apache.org>.
Carsten Ziegeler wrote:
> > Paul Russell wrote:
> >
> > Yeah, that's fine, but a very strong -1 for just adding classes
> > because they 'might be useful'. If the cache needs them, then
> > that's fine, if something else *core* needs them, then that's fine,
> > but Cocoon2 is already 23,000 lines of code, let's not make it bigger
> > than it has to be. For those familiar with XP, You Ain't Gonna Need it.
> > For those who aren't <URL:http://www.extremeprogramming.org/> is *well*
> > worth a read, it might just change the way you work.
>
> Believe me, I am familiar with XP....Of course, you are right: just adding
> classes which are never be used by the project itself, should be avoided.
>
> OK, I didn't want to come up with this at this stage, but e.g. I could
> integrate the XMLCompiler into the FileGenerator for caching. So the
> FileGenerator can cache the parsed file using the XMLCompiler. This would
> speed up the FileGenerator (I think Stefano proposed that when he announced
> the XMLCompiler). This caching would be independent from the big caching
> algorithm C2 uses, like the caching of stylesheets in the XalanTransformer.
> In addition the big - yet-to-do - caching algorithm might need it, to cache
> partial responses. I don't mean caching at each stage in the pipeline!
> Sometimes it might be usefull to cache a response upto a specific point in
> the pipeline as all following stages are highly dynamic - but this is just
> a my thoguhts...
Ok, this sounds much better.
Don't mix cache and store. Cache is the thing we want to speed up processing
almost transparent to each component. The FileGenerator (when it uses the
XMLCompiler) can use the general component Store (as the Xalantransformer is
doing) to store compiled xml files into. Integrating the XMLCompiler into the
Filegenerator should be a snap, isn't it?
Giacomo
AW: Status of XMLCompiler - Please vote...
Posted by Carsten Ziegeler <cz...@sundn.de>.
> Paul Russell wrote:
>
> Yeah, that's fine, but a very strong -1 for just adding classes
> because they 'might be useful'. If the cache needs them, then
> that's fine, if something else *core* needs them, then that's fine,
> but Cocoon2 is already 23,000 lines of code, let's not make it bigger
> than it has to be. For those familiar with XP, You Ain't Gonna Need it.
> For those who aren't <URL:http://www.extremeprogramming.org/> is *well*
> worth a read, it might just change the way you work.
>
Believe me, I am familiar with XP....Of course, you are right: just adding
classes which are never be used by the project itself, should be avoided.
OK, I didn't want to come up with this at this stage, but e.g. I could
integrate the XMLCompiler into the FileGenerator for caching. So the
FileGenerator can cache the parsed file using the XMLCompiler. This would
speed up the FileGenerator (I think Stefano proposed that when he announced
the XMLCompiler). This caching would be independent from the big caching
algorithm C2 uses, like the caching of stylesheets in the XalanTransformer.
In addition the big - yet-to-do - caching algorithm might need it, to cache
partial responses. I don't mean caching at each stage in the pipeline!
Sometimes it might be usefull to cache a response upto a specific point in the
pipeline as all following stages are highly dynamic - but this is just a
my thoguhts...
> > (OK, I know, I can't start a vote - but it's worth trying...)
>
> Hahaha. We're open source. You can start what you like :)
>
Really? Ok, ehm.....ehm....I want to start integrating the XMLCompiler into C2! Is this OK.....
Carsten
Re: AW: Status of XMLCompiler - Please vote...
Posted by Paul Russell <pa...@luminas.co.uk>.
* Carsten Ziegeler (cziegeler@sundn.de) wrote :
> Integrating the XMLCompiler doesn't hurt at all as no existing
> component is affect, no interface has to change - simply nothing,
> just adding classes.
Yeah, that's fine, but a very strong -1 for just adding classes
because they 'might be useful'. If the cache needs them, then
that's fine, if something else *core* needs them, then that's fine,
but Cocoon2 is already 23,000 lines of code, let's not make it bigger
than it has to be. For those familiar with XP, You Ain't Gonna Need it.
For those who aren't <URL:http://www.extremeprogramming.org/> is *well*
worth a read, it might just change the way you work.
> (OK, I know, I can't start a vote - but it's worth trying...)
Hahaha. We're open source. You can start what you like :)
Paul
--
Paul Russell Email: paul@luminas.co.uk
Technical Director Tel: +44 (0)20 8553 6622
Luminas Internet Applications Fax: +44 (0)870 28 47489
This is not an official statement or order. Web: www.luminas.co.uk