You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Stefano Mazzocchi <st...@apache.org> on 2005/10/06 21:06:36 UTC
Cocoon Fat Test
Carsten hates me when I "just talk", so here is some action: let's
measure cocoon's fat.
Follow me and type the following in your terminal
svn co http://svn.apache.org/repos/asf/cocoon/branches/BRANCH_2_1_X/ \
cocoon
svn co http://simile.mit.edu/repository/linotype/trunk/ linotype
export COCOON_HOME=../cocoon/
cd linotype
ant install
ant run
then point your browser at http://127.0.0.1:8888/linotype/
you should see an empty linotype running.
Now, ctrl-break the above, and do
zip -r linotype.war dist/
the resulting WAR is 12263572 bytes. How much of this is linotype?
zip -r linotype.zip dist/linotype/
results in a 129673 bytes. Cocoon has a platform overhead of 11Mb.
*compressed*
Now let's see where the things are:
cd dist
du -d1
returns
1152 ./linotype
144 ./resources
128 ./stylesheets
27472 ./WEB-INF
29000 .
so
cd WEB-INF
du -d1
returns
8 ./classes
1544 ./entities
25776 ./lib
27472 .
So, it's clear that cocoon weight is in the jars.
NOTE: the linotype build system builds cocoon from source with the most
stripped-down version as Linotype does not depend on any blocks. This is
the minimal 2.1.x cocoon possible right now.
Let's see the list of jars
cd lib
ls -AlS
(with some little hand cleaning)
2941481 jdtcore-3.0.2.jar
2730442 xalan-2.7.0.jar
1395202 cocoon-2.1.8-dev.jar
1203860 xercesImpl-2.7.1.jar
607235 rhino1.5r4-continuations-R26.jar
552263 commons-collections-3.1.jar
522143 jakarta-bcel-20040329.jar
394671 jcs-1.2.5-dev-20050313.jar
358085 log4j-1.2.12.jar
285104 commons-jxpath-1.2.jar
225375 commons-httpclient-2.0.2.jar
207723 commons-lang-2.1.jar
194205 xml-apis-1.3.02.jar
189284 util.concurrent-1.3.4.jar
131458 commons-jexl-1.0.jar
115506 excalibur-instrument-mgr-http-2.1.jar
89145 excalibur-logger-2.1.jar
86665 excalibur-xmlutil-2.1.jar
83582 avalon-logkit-2.1.jar
80572 excalibur-component-1.2.jar
78599 excalibur-sourceresolve-2.1.jar
65948 excalibur-naming-1.0.jar
60047 xml-commons-resolver-1.1.jar
59645 avalon-framework-impl-4.3.jar
57548 excalibur-instrument-mgr-impl-2.1.jar
47531 ehcache-1.1.jar
42137 excalibur-store-2.1.jar
40737 excalibur-io-1.1.jar
38015 commons-logging-1.0.4.jar
32112 avalon-framework-api-4.3.jar
30117 commons-cli-1.0.jar
28584 jakarta-regexp-1.4.jar
27833 excalibur-pool-impl-2.1.jar
21028 javacImpl-0.9.jar
20640 excalibur-instrument-mgr-api-2.1.jar
20328 excalibur-instrument-api-2.1.jar
18329 excalibur-pool-instrumented-2.1.jar
14728 excalibur-pool-api-2.1.jar
8238 excalibur-i18n-1.1.jar
3565 javacApi-0.9.jar
now, let's take a look
1) I don't use XSP nor javaflow, therefore I wouldn't need JDTcore,
nor BCEL, nor javacImpl and javacAPI, a total of several Mb.
2) I don't use the avalon instrumentation
3) I use logkit, so I don't need log4j
4) I put the latest xalan and xerces in my app server, no need to have
it here too
here is the slimmed down cocoon
1395202 cocoon-2.1.8-dev.jar
607235 rhino1.5r4-continuations-R26.jar
552263 commons-collections-3.1.jar
285104 commons-jxpath-1.2.jar
225375 commons-httpclient-2.0.2.jar
207723 commons-lang-2.1.jar
194205 xml-apis-1.3.02.jar
189284 util.concurrent-1.3.4.jar
131458 commons-jexl-1.0.jar
89145 excalibur-logger-2.1.jar
86665 excalibur-xmlutil-2.1.jar
83582 avalon-logkit-2.1.jar
80572 excalibur-component-1.2.jar
78599 excalibur-sourceresolve-2.1.jar
65948 excalibur-naming-1.0.jar
60047 xml-commons-resolver-1.1.jar
59645 avalon-framework-impl-4.3.jar
47531 ehcache-1.1.jar
42137 excalibur-store-2.1.jar
40737 excalibur-io-1.1.jar
38015 commons-logging-1.0.4.jar
32112 avalon-framework-api-4.3.jar
30117 commons-cli-1.0.jar
28584 jakarta-regexp-1.4.jar
27833 excalibur-pool-impl-2.1.jar
18329 excalibur-pool-instrumented-2.1.jar
14728 excalibur-pool-api-2.1.jar
8238 excalibur-i18n-1.1.jar
which now yields a war file of 4Mb, still a lot but much better than before.
Anyway, I know some of this has been discussed at lenght on the mail
list already, so I tried cocoon 2.2.x to see if things are improving,
let's redo the little dance:
svn co http://svn.apache.org/repos/asf/cocoon/trunk/ cocoon
svn co http://simile.mit.edu/repository/linotype/trunk/ linotype
export COCOON_HOME=../cocoon/
cd linotype
ant war
results in a 11516061 file, which is better than before, well, let's see
if it works
ant run
does cocoon work? [open http://127.0.0.1:8888/] yes it does.
does linotype work? [open http://127.0.0.1:8888/linotype/] nope
"Type 'jx' does not exist for 'map:transform' at", let's see the sitemap
<map:components>
<!-- include some additional components -->
<map:include dir="context://WEB-INF/sitemap-additions"
pattern="*.xconf"/>
</map:components>
ah, ok, cool, now the components are located somewhere else, let's see
the sitemap-additions
<map:components xmlns:map="http://apache.org/cocoon/sitemap/1.0">
<map:generators>
<map:generator label="content"
logger="sitemap.generator.mp3directory" name="mp3directory"
src="org.apache.cocoon.generation.MP3DirectoryGenerator"/>
</map:generators>
<map:transformers>
<map:transformer logger="sitemap.transformer.jpath" name="jpath"
src="org.apache.cocoon.transformation.JPathTransformer"/>
<map:transformer logger="sitemap.transformer.filter" name="filter"
src="org.apache.cocoon.transformation.FilterTransformer"/>
</map:transformers>
<map:serializers>
<map:serializer logger="sitemap.serializer.vrml"
mime-type="model/vrml" name="vrml"
src="org.apache.cocoon.serialization.TextSerializer"/>
</map:serializers>
</map:components>
ehmmm, all right, ... but where did the standard sitemap components go?
[sound of stefano searching] oh, here they are
context://WEB-INF/xconf/cocoon-core-sitemap.xconf
but the main sitemap doesn't link them... didn't we agree not to have
black magic in the sitemaps?
Not sure I don't like this, reduces discoverability a great deal. But
anyway, let's look for the "jx" transformer... hmmm, gone, all right,
it's not that core anyway... hmmm, oh, here it is, it's the 'template'
block, that wasn't there in 2.1.x, let's add this to the build system.
So, let's add -Dinclude.block.template=true when invoking ant....
run.... nope.
So, -Dinclude.block.template=true does not overwrite
-Dexclude.blocks.all=true. Boy, that's lame.
The ability to exclude or include all blocks is a great one, but if
individual selections don't override the default, it's a problem.
Anyway, that can be fixed.
So, here is the diff between the dist-2.1.x/WEB-INF/lib and the
dist-2.2.x/WEB-INF/lib with no blocks enabled
Only in dist-2.1.x/WEB-INF/lib/: cocoon-2.1.8-dev.jar
Only in dist-2.1.x/WEB-INF/lib/: excalibur-component-1.2.jar
Only in dist-2.1.x/WEB-INF/lib/: excalibur-instrument-mgr-api-2.1.jar
Only in dist-2.1.x/WEB-INF/lib/: excalibur-instrument-mgr-http-2.1.jar
Only in dist-2.1.x/WEB-INF/lib/: excalibur-instrument-mgr-impl-2.1.jar
Only in dist-2.1.x/WEB-INF/lib/: javacApi-0.9.jar
Only in dist-2.1.x/WEB-INF/lib/: javacImpl-0.9.jar
Only in dist-2.1.x/WEB-INF/lib/: jdtcore-3.0.2.jar
Only in dist-2.1.x/WEB-INF/lib/: rhino1.5r4-continuations-R26.jar
Only in dist-2.2.x/WEB-INF/lib/: cocoon-2.2.0-dev.jar
Only in dist-2.2.x/WEB-INF/lib/: commons-javaflow-r306555.jar
Only in dist-2.2.x/WEB-INF/lib/: commons-jci-r306555.jar
Only in dist-2.2.x/WEB-INF/lib/: knopflerfish-cm_api-1.0.0.jar
Only in dist-2.2.x/WEB-INF/lib/: knopflerfish-console_all-1.0.0.jar
Only in dist-2.2.x/WEB-INF/lib/: knopflerfish-consoletty-1.0.0.jar
Only in dist-2.2.x/WEB-INF/lib/: knopflerfish-framework-1.3.3.jar
Only in dist-2.2.x/WEB-INF/lib/: knopflerfish-frameworkcommands-1.0.0.jar
Only in dist-2.2.x/WEB-INF/lib/: knopflerfish-http_all-1.1.0.jar
Only in dist-2.2.x/WEB-INF/lib/: knopflerfish-log_all-1.0.0.jar
Only in dist-2.2.x/WEB-INF/lib/: knopflerfish-logcommands-1.0.0.jar
Only in dist-2.2.x/WEB-INF/lib/: rhino-1.6R2.jar
Only in dist-2.2.x/WEB-INF/lib/: xmlbeans-1.0.3.jar
So, let's see how much I can strip it down and keep the welcome page
show up:
1) let's remove knopflerfish -> works
2) let's remove xalan, xerces and the xml api -> works
3) let's remove javaflow -> works
4) let's remove log4j -> works
5) let's remove bcel -> works
6) let's remove xmlbeans -> works
7) let's remove jcs -> works
so, how much are we now? 4Mb the total war file. Not so bad.
Where can we improve? well, cocoon.jar is about 1Mb, compressed, that
seems like a lot of stuff, let's see what's inside
cd dist/WEB-INF/lib
unzip cocoon-2.2.0.jar
cd org/apache/cocoon
du -d1 | sort -rn
3936 ./components
976 ./transformation
688 ./xml
664 ./core
632 ./environment
520 ./util
472 ./generation
360 ./acting
312 ./matching
272 ./sitemap
192 ./selection
192 ./bean
160 ./servlet
144 ./serialization
128 ./i18n
80 ./reading
72 ./caching
24 ./configuration
cd components
du -d1 | sort -rn
1040 ./treeprocessor
728 ./modules
528 ./flow
352 ./source
240 ./xpointer
208 ./pipeline
144 ./thread
136 ./store
112 ./blocks
88 ./sax
72 ./notification
72 ./classloader
56 ./crawler
24 ./persistence
24 ./fam
24 ./container
16 ./resolver
hmmm, not a lot to remove, treeprocessor + flow + transformation occupy
pretty much all of it.
With a serious diet and an improved build system, cocoon could lose half
of its weight and generate webapps that are much more friendly for
smaller sites or for inclusion in places where size is an issue.
But even when size is not an issue, having smaller webapps helps in
making users perceive what core functionality the cocoon framework gives
them and what is add-on, something that we were never very good at
marketing.
Hope this helps.
--
Stefano.
Re: Cocoon Fat Test
Posted by Mark Lundquist <ml...@wrinkledog.com>.
On Oct 7, 2005, at 1:00 PM, Leszek Gawron wrote:
> I do not get it. It's like saying "if you forget to tank your car will
> stop - how annoying" :). What I mean is that the same problem applies
> to any block (i.e. forms). It's just jx and forms tend to be most
> "core" ones.
>
> I think it was a good decision to move jx from core to it's own block.
>
I agree.
—ml—
Re: Cocoon Fat Test
Posted by Stefano Mazzocchi <st...@apache.org>.
Leszek Gawron wrote:
> Carsten Ziegeler wrote:
>
>> And I agree, not having the "jx" transformer/generator in the core
>> anymore is really annoying. If you forget to include the template block
>> it breaks your application immediately. But I hope we will fix this, as
>> well.
>
> I do not get it. It's like saying "if you forget to tank your car will
> stop - how annoying" :). What I mean is that the same problem applies to
> any block (i.e. forms). It's just jx and forms tend to be most "core" ones.
>
> I think it was a good decision to move jx from core to it's own block.
>
> It's a stupid work to customize cocoon blocks - this is the biggest
> problem. I agree with Stefano. My local.blocks.properties should state:
>
> exclude.all.blocks=true
> include.block.forms=true
> include.block.template=true
>
> that's all. Someone said some time ago to use :s/#include/include. Try
> to merge your customized local.blocks.properties with updated main
> blocks.properties file.
Agreed. I never said it was annoying to move stuff out, I don't mind to
have to specify what blocks to add, but I want this to be easy not a
PITA like it is today.
My local.blocks.properties should look like Leszek wrote above. Easy and
simple. That would make us go a long way forward.
--
Stefano.
Re: Cocoon Fat Test
Posted by Leszek Gawron <lg...@mobilebox.pl>.
Carsten Ziegeler wrote:
> And I agree, not having the "jx" transformer/generator in the core
> anymore is really annoying. If you forget to include the template block
> it breaks your application immediately. But I hope we will fix this, as
> well.
I do not get it. It's like saying "if you forget to tank your car will
stop - how annoying" :). What I mean is that the same problem applies to
any block (i.e. forms). It's just jx and forms tend to be most "core" ones.
I think it was a good decision to move jx from core to it's own block.
It's a stupid work to customize cocoon blocks - this is the biggest
problem. I agree with Stefano. My local.blocks.properties should state:
exclude.all.blocks=true
include.block.forms=true
include.block.template=true
that's all. Someone said some time ago to use :s/#include/include. Try
to merge your customized local.blocks.properties with updated main
blocks.properties file.
--
Leszek Gawron lgawron@mobilebox.pl
IT Manager MobileBox sp. z o.o.
+48 (61) 855 06 67 http://www.mobilebox.pl
mobile: +48 (501) 720 812 fax: +48 (61) 853 29 65
Re: Cocoon Fat Test
Posted by Carsten Ziegeler <cz...@apache.org>.
Stefano Mazzocchi wrote:
> <snip/>
>
> With a serious diet and an improved build system, cocoon could lose half
> of its weight and generate webapps that are much more friendly for
> smaller sites or for inclusion in places where size is an issue.
>
> But even when size is not an issue, having smaller webapps helps in
> making users perceive what core functionality the cocoon framework gives
> them and what is add-on, something that we were never very good at
> marketing.
>
> Hope this helps.
>
Yepp, we are currently working on exactly this with the maven build of
2.2. So with the m2 build system we will not ship libs that we just need
for compilation anymore, like our version of Xalan and so on.
We will also strip down the core, reducing the size and the required
libs even more. And this then will lead to different distributions, like
a minimum core dist, a big "show us everything one", and perhaps more.
And finally, building your webapp will be easy as well, you will just
say, "include this block" in your maven descriptor and the rest is done
by maven.
So, we know where we want to go, we know how it can be done, there are
no real technical problems anymore. All is missing, is just doing it :)
And I agree, not having the "jx" transformer/generator in the core
anymore is really annoying. If you forget to include the template block
it breaks your application immediately. But I hope we will fix this, as
well.
Carsten
--
Carsten Ziegeler - Open Source Group, S&N AG
http://www.s-und-n.de
http://www.osoco.org/weblogs/rael/
Re: Cocoon Fat Test
Posted by Carsten Ziegeler <cz...@apache.org>.
Niclas Hedhman wrote:
>
> However, to me "fatness" in distribution is somewhat moot point nowadays. I
> have had some concerns over the runtime RAM footprint, but have no conclusive
> numbers, whether it is a leak problem, or just caching going nuts. Scheduled
> server restarts doesn't sound like a good solution, but at the moment it is
> the 'cheapest' thing to do.
>
Usually there are two reasons for this: the memory cache is becomming to
big or you have a memory leak :) The first one can be handled easily by
configuring the store janitor correctly. As soon as you have configured
this one correctly, the only reason for an out of memory exception is a
real memory leak.
I think if you're using the ordinary Cocoon stuff, we are not aware of
any memory leaks. We did search memory leaks for many customers and most
time, the memory leak was in there own code which we could find very easily.
Carsten
--
Carsten Ziegeler - Open Source Group, S&N AG
http://www.s-und-n.de
http://www.osoco.org/weblogs/rael/
Re: Cocoon Fat Test
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Friday 07 October 2005 14:15, Carsten Ziegeler wrote:
> so I think you're trying to put words into my mouth.
Maybe you hate him for that ;o)
> Ok, now let's get back to the technical discussion...
Stefano Mazzocchi wrote:
> > But even when size is not an issue, having smaller webapps helps in
> > making users perceive what core functionality the cocoon framework gives
> > them and what is add-on.
Indeed. An easy-to-understand "fairly simple" system with 50 plug-ins looks
like a better deal to a newcomer than a big monolith, which is hard to grasp.
However, to me "fatness" in distribution is somewhat moot point nowadays. I
have had some concerns over the runtime RAM footprint, but have no conclusive
numbers, whether it is a leak problem, or just caching going nuts. Scheduled
server restarts doesn't sound like a good solution, but at the moment it is
the 'cheapest' thing to do.
Cheers
Niclas
Re: Cocoon Fat Test
Posted by Carsten Ziegeler <cz...@apache.org>.
Stefano Mazzocchi wrote:
> Carsten hates me when I "just talk", so here is some action: let's
> measure cocoon's fat.
>
LOL - I really hope that you don't mean this that seriously.
Just to be clear, I don't hate anyone just because he is not doing
something or is doing the "wrong" thing or whatever. There is noone in
our community I hate, so I think you're trying to put words into my
mouth. And it's good that we all have different opinions from time to
time. It would be so borring if we would agree always on everything
immediately.
Anyways, there is the good old saying (which I'm actually not really a
fan of): "sometimes you have to provoke to reach a reaction." So I guess
some of us (including myself and you) are sometimes trying it this way.
;) And it's not surprising that sometimes this works and sometimes you
get a reaction you really didn't expect. That makes life interesting and
I guess/hope we all really can deal with this in an adult fashion.
Ok, now let's get back to the technical discussion...
Carsten
--
Carsten Ziegeler - Open Source Group, S&N AG
http://www.s-und-n.de
http://www.osoco.org/weblogs/rael/
Re: Cocoon Fat Test
Posted by Daniel Fagerstrom <da...@nada.kth.se>.
Stefano Mazzocchi wrote:
...
Carsten and Leszek addressed most of the other points.
> ehmmm, all right, ... but where did the standard sitemap components
> go? [sound of stefano searching] oh, here they are
>
> context://WEB-INF/xconf/cocoon-core-sitemap.xconf
>
> but the main sitemap doesn't link them... didn't we agree not to have
> black magic in the sitemaps?
It is linked from cocoon.xconf, so there are no black magic, yet.
The reason for this change is that for real blocks we must declare in
some way what components a block exposes to blocks that depend on it. So
this means that all exposed components should be configured in one
place. This could either be a block global component configuration, like
cocoon.xconf or in the main sitemap. As blocks doesn't have to have a
sitemap, they can be component only, it doesn't make that much sense to
always require a sitemap. We decided to move the exposed sitemap
components to a block.xconf because of that. All of this has been
discussed on the list.
There will still not be any black magic as both the block component
configuration and the block sitemap are refered to from block.xml. OTH
it might be convenient to have some default locations for component
configuration and sitemap in blocks.
At a later stage you might need to look even further for the component
configuration. IMO components should in general be defined in the block
that contain the implementation of them. In this way we only need to
make interfaces exported from blocks, implementation classes can be
internal. From this it would be more natural to consider core (src/java)
as a block and put the configuration of the core components there
instead on in src/webapp. This would follow the pattern from all other
blocks.
...
> So, let's see how much I can strip it down and keep the welcome page
> show up:
>
> 1) let's remove knopflerfish -> works
Knopflerfish is the OSGi implementation, only o.a.c.core.osgi in core
and src/osgi-servlet depends on these. The libraries will be moved to
lib/osgi and the o.a.c.core.osgi package will be moved to an own block,
to make it easier to exclude the OSGi stuff from trunk while releaseing
2.2.x. (I organized things like this from the beginning, but someone
reorganized it).
> 2) let's remove xalan, xerces and the xml api -> works
> 3) let's remove javaflow -> works
We could also remove flowscripts to an own block. It doesn't help you,
but would remove the fat from Rhino for those not using it. I don't
think anything in the rest of core depends on the flowscript
implementation anymore.
> 4) let's remove log4j -> works
Move to own block (or bundle). I think we should start to think in terms
of library bundles (or blocks) also, i.e. blocks that just contain java
code and/or resources, but doesn't expose components or sitemap
functionality. We don't need to wait for start using OSGi for this. We
could make it possible to switch on and off optional libraries as well
as blocks.
> 5) let's remove bcel -> works
> 6) let's remove xmlbeans -> works
xmlbeans is (or at least was), only a runtime dependency for making it
possible to use E4X in flowscripts. I should definitely become an
optional library.
> 7) let's remove jcs -> works
>
> so, how much are we now? 4Mb the total war file. Not so bad.
>
> Where can we improve? well, cocoon.jar is about 1Mb, compressed, that
> seems like a lot of stuff, let's see what's inside
>
> cd dist/WEB-INF/lib
> unzip cocoon-2.2.0.jar
> cd org/apache/cocoon
> du -d1 | sort -rn
>
> 3936 ./components
> 976 ./transformation
> 688 ./xml
> 664 ./core
> 632 ./environment
> 520 ./util
> 472 ./generation
> 360 ./acting
> 312 ./matching
> 272 ./sitemap
> 192 ./selection
> 192 ./bean
> 160 ./servlet
> 144 ./serialization
> 128 ./i18n
> 80 ./reading
> 72 ./caching
> 24 ./configuration
>
> cd components
> du -d1 | sort -rn
>
> 1040 ./treeprocessor
> 728 ./modules
> 528 ./flow
> 352 ./source
> 240 ./xpointer
> 208 ./pipeline
> 144 ./thread
> 136 ./store
> 112 ./blocks
> 88 ./sax
> 72 ./notification
> 72 ./classloader
> 56 ./crawler
> 24 ./persistence
> 24 ./fam
> 24 ./container
> 16 ./resolver
>
> hmmm, not a lot to remove, treeprocessor + flow + transformation
> occupy pretty much all of it.
The sitemap components and some of the ordinary components. e.g. most of
the modules, could preferably be moved to separate blocks as well.
It doesn't matter that much from "fatness" POV. But when we start with
separate release cycles for all blocks, it will be an advantage to keep
the "core" as small and focused as possible. That will make it obvious
what is the core contracts, and it will also keep us diciplined about
dependencies to core.
> With a serious diet and an improved build system, cocoon could lose
> half of its weight and generate webapps that are much more friendly
> for smaller sites or for inclusion in places where size is an issue.
>
> But even when size is not an issue, having smaller webapps helps in
> making users perceive what core functionality the cocoon framework
> gives them and what is add-on, something that we were never very good
> at marketing.
Yes, and will also make it easier for people to start developing Cocoon,
if we make the core lean. Now it is quite a huge amount of code to
search in, when you try to understand the core.
/Daniel
Re: Cocoon Fat Test
Posted by Torsten Curdt <tc...@apache.org>.
> With a serious diet and an improved build system, cocoon could lose
> half of its weight and generate webapps that are much more friendly
> for smaller sites or for inclusion in places where size is an issue.
>
> But even when size is not an issue, having smaller webapps helps in
> making users perceive what core functionality the cocoon framework
> gives them and what is add-on, something that we were never very
> good at marketing.
...one of the things I gonna tell the people on my talk (aside):
strip down the baby - otherwise it's a pain. It would be
more than nice if the build system would take care of that.
Maybe even http://proguard.sourceforge.net can help with it
...but in theory our build system should be smart enough
to take care of this.
AND ...I am way in favor of shipping with a minimal installation
instead of a "cocoon-can-everything-lets-show-it" configuration.
We passed that size. TBH I think the complexity more scares people
away than it helps.
Let's get them a clean and simple ...minimal sitemap to start
with and then improve the documentation "you need this? - add that"
My 2 cents
--
Torsten