Cocoon in Business Environements and its shortfalls

[Robert Simmons <ro...@norcom.de>]

I wanted to open a general discussion thread on the shortfalls of Cocoon in
business environments. I have some very clear opinions on the matter, but
then my opinions are just that -- opinions. Others may differ with me.

First of all, I should say that I have a very direct, a-political, style
that sometimes treads on toes unintentionally. Trust me when I say that if I
thought Cocoon was a piece of junk, I would say so. As a matter of fact I
think it is an excellent product with some shortcommings like all products
have. However, Im not particularly into "ra-ra" chearleading so I tend to
only point out things I see as deficiencies. In addition, Im not opposed to
putting my time where my mouth is so long as what I would contribute is
feasible without 2 months of study to understand the source base.

Back to the subject at hand. If we analyze the shortcommings in a business
environment, we can potentially plan out strategies to make Cocoon leap from
a sometimes used app to something as ubiquitous as ant or Apache web server.
This is a leap from pure open source to business ready open source and is
not an easy one.

I will start by stating my issues with cocoon as is:

1) Production builds are lacking. What I mean by this are builds that a
developer would use if he is USING cocoon and not developing on it at all.

1a) One might object that blocks have to be described dureing the build
process. This could be true but needs to change. The ideal solution is that
someone cha drop in a block into the blocks directory post build and then
indicate the block in the sitemap. This would allow a binary build that
would give users the chance to merely delete blocks that they are not
interested in and add their own blocks to a binary build.

1b) Examples for blocks need to be out of the blocks themselves. In my
opinion, the only thing in the blocks directory should be the blocks jars
themselves and perhaps something akin to a deployment descriptor for the
blocks in the directory. Examples should be in a completely different
locations.

2) Monitoring is not intuitive. If you are deploying a business application
with 20 cocurrent Cocoon instances, you need a way to cohesively monitor the
health of the entire cluster. Separate log files just arent sufficient.
Something like JMX instrumentation would be ideal.

3) Cocoon needs cluster management and load balancing functionality. You
should be able to set up a cluster of cocoon instances that know about each
other and have them perform load balancing and failover. This needs to be
established at the cocoon level because the servlet engine wont have
information about how busy cocoon is. For example, a cocoon instance getting
a single reuest per second might be more busy than one getting 20 requests
per second because that single request takes lots more processing time and
resources.

4) Cocoon documentation is minimal and non-cohesive. (But you knew this
already) Although wiki is good for knowledge sharing, the information in
Wiki is oftewn out of date and its reliability is sometimes questionable.

5) Performance. Cocoon needs to have some routines gone over with a fine
tooth comb for performance reasons. Think of deploying cocoon in a 100
concurrent user environment and having it take the load. Things Id like to
see include URL per-caching at startup, pipeline timing for performance
tuning and component execution timing. If these items were instrumented with
JMX it would be even better.

Issues Im not privy to directly but concerned about:

6) Project Lint. Over time any project accumulates an amount of lint in the
form of dead code, unused classes and unused methods. How much of this is in
cocoon and how can we get it out?

Comments? Additions? Suggestions? (Please route flames to /dev/null).

-- Robert

Re: Cocoon in Business Environements and its shortfalls

[Berin Loritsch <bl...@apache.org>]

Robert Simmons wrote:


*Snip many good points*

> Issues Im not privy to directly but concerned about:
> 
> 6) Project Lint. Over time any project accumulates an amount of lint in the
> form of dead code, unused classes and unused methods. How much of this is in
> cocoon and how can we get it out?
> 

I have found that trying to use tools to verify the "lint" in a project with
true separation of components and interfaces, the method coverage metrics
generate a bunch of false positives.  You will find that all your Generators
won't be directly connected to in code, so they will be tagged unused code.
You might be able to find which methods on an interface are not used, but
tools for environments like this are not advanced enough yet.

Nevertheless, certain IDEs like JetBrains IDEA is very good at finding dead
private methods, unused variables, unused parameters (and determining if the
method is from an interface or not), etc.  I wish I was aware of an OS project
that was that advanced.

-- 

"They that give up essential liberty to obtain a little temporary safety
  deserve neither liberty nor safety."
                 - Benjamin Franklin

Re: Cocoon in Business Environements and its shortfalls

[Geoff Howard <co...@leverageweb.com>]

Robert Simmons wrote:
> I wanted to open a general discussion thread on the shortfalls of Cocoon in
> business environments. I have some very clear opinions on the matter, but
> then my opinions are just that -- opinions. Others may differ with me.

...

> I will start by stating my issues with cocoon as is:
> 
> 1) Production builds are lacking. What I mean by this are builds that a
> developer would use if he is USING cocoon and not developing on it at all.

Robert, I think you are making the (understandable) assumption that 
getting rid of the binary build betrayed a focus on people developing on 
Cocoon itself, but this is incorrect.  Have you found the recent (last 
week or two) comments I made explaining this on the users list?  I have 
had several people with the same initial feeling you're expressing write 
back to me privately after that indicating that the current (temporary) 
solution is not nearly as bad as they perceived it to be.

To restate simply, the build step really is a "configure" step and a 
build step rolled into one.  We could probably (now that the JDK 
dependencies are fixed) distribute a "half built" Cocoon, where all 
classes are compiled but the webapp is not configured/assembled.  That 
would not get you out of having to run ./something.sh target-name to do 
the last step.  So it has been the judgement of the community that since 
that's necessary anyway, there is not a big functional difference 
between the two and what we have can do for now.  We could probably 
rename build.sh to configure.sh and remove the bulk of the perception 
problem.

> 1a) One might object that blocks have to be described dureing the build
> process. This could be true but needs to change. The ideal solution is that
> someone cha drop in a block into the blocks directory post build and then
> indicate the block in the sitemap. This would allow a binary build that
> would give users the chance to merely delete blocks that they are not
> interested in and add their own blocks to a binary build.

You need to note that we already have an excellent solution in the works 
which will allow binary drop-in of new components/applications/services 
into a pre-built binary core.  I believe this will exceed most people's 
expectations.

We just released 2.1.  Planning for the next step has been in the works 
for probably a year and execution is beginning now.

> 1b) Examples for blocks need to be out of the blocks themselves. In my
> opinion, the only thing in the blocks directory should be the blocks jars
> themselves and perhaps something akin to a deployment descriptor for the
> blocks in the directory. Examples should be in a completely different
> locations.

Are you aware that including a block with exclude.webapp.samples (or 
whatever) configured excludes the examples for that block already? 
Given that, I completely disagree that the samples should be moved 
somewhere else.  A block (as it exists now, not in the full 2.2 
implementation under development now) is not a discrete thing that can 
be picked up and deleted.  It's an aggregation of java classes and 
configuration, and (optionally) samples.

...

> 4) Cocoon documentation is minimal and non-cohesive. (But you knew this
> already) Although wiki is good for knowledge sharing, the information in
> Wiki is oftewn out of date and its reliability is sometimes questionable.

Unfortunately, so is the "official" documentation in places.  Newcomers 
who don't feel comforatble writing new docs could still help immensely 
in giving good detailed feedback about what is missing, what is out of 
date or questionable, and what should move from the wiki to the official 
docs.

Geoff

Re: Cocoon in Business Environements and its shortfalls

[Berin Loritsch <bl...@apache.org>]

Robert Simmons wrote:


> Back to the subject at hand. If we analyze the shortcommings in a business
> environment, we can potentially plan out strategies to make Cocoon leap from
> a sometimes used app to something as ubiquitous as ant or Apache web server.
> This is a leap from pure open source to business ready open source and is
> not an easy one.
> 
> I will start by stating my issues with cocoon as is:
> 
> 1) Production builds are lacking. What I mean by this are builds that a
> developer would use if he is USING cocoon and not developing on it at all.

I think this comes from simplifying Cocoon to its core, and making blocks
truly work.

> 1a) One might object that blocks have to be described dureing the build
> process. This could be true but needs to change. The ideal solution is that
> someone cha drop in a block into the blocks directory post build and then
> indicate the block in the sitemap. This would allow a binary build that
> would give users the chance to merely delete blocks that they are not
> interested in and add their own blocks to a binary build.

I think we are in agreement on this.

> 1b) Examples for blocks need to be out of the blocks themselves. In my
> opinion, the only thing in the blocks directory should be the blocks jars
> themselves and perhaps something akin to a deployment descriptor for the
> blocks in the directory. Examples should be in a completely different
> locations.

We need to build the blocks.  Where would you suggest putting the source code?

> 2) Monitoring is not intuitive. If you are deploying a business application
> with 20 cocurrent Cocoon instances, you need a way to cohesively monitor the
> health of the entire cluster. Separate log files just arent sufficient.
> Something like JMX instrumentation would be ideal.

Understood.  Keep in mind that Cocoon does have some instrumentation available
to it through Avalon's instrumentation package.  It has a nice client that can
connect to remote servers.  That way you can monitor your Cocoon instances from
your desk, and see their relative happiness.

> 3) Cocoon needs cluster management and load balancing functionality. You
> should be able to set up a cluster of cocoon instances that know about each
> other and have them perform load balancing and failover. This needs to be
> established at the cocoon level because the servlet engine wont have
> information about how busy cocoon is. For example, a cocoon instance getting
> a single reuest per second might be more busy than one getting 20 requests
> per second because that single request takes lots more processing time and
> resources.

Clustering is really an issue for the container.  For all intents and purposes,
it is a Servlet.  It load balances as well as the Servlet Container does.
I would focus on the blocks first, and then address this issue.

> 4) Cocoon documentation is minimal and non-cohesive. (But you knew this
> already) Although wiki is good for knowledge sharing, the information in
> Wiki is oftewn out of date and its reliability is sometimes questionable.

Hmm.  Is there any way to make it more self-documenting?

> 5) Performance. Cocoon needs to have some routines gone over with a fine
> tooth comb for performance reasons. Think of deploying cocoon in a 100
> concurrent user environment and having it take the load. Things Id like to
> see include URL per-caching at startup, pipeline timing for performance
> tuning and component execution timing. If these items were instrumented with
> JMX it would be even better.

In the mean time it would be easier to instrument it with the Avalon
instrumentation package.  JMX integration is planned for incorporation into
Avalon containers--but it just isn't here yet.

-- 

"They that give up essential liberty to obtain a little temporary safety
  deserve neither liberty nor safety."
                 - Benjamin Franklin