Core split status

[Daniel Fagerstrom <da...@nada.kth.se>]

While the split of the core is far from finished, some structure is 
starting to emerge. So I'll describe what we have done, where we are 
going and open issues so that more people can participate in the work 
and discussions.

Some background
===============

The aim with the activity is to make components that are part of Cocoon 
reusable outside Cocoon. And to make the architecture of Cocoon more 
clearly layered. The later to make Cocoon easier to understand and 
maintain. The main layers are component management, pipelines and sitemaps.

To make a component reusable outside the component should preferable be 
a POJO that has minimal dependencies and where the dependencies are 
stable and from well established projects. The second best alternative 
if the component cannot be a pure POJO is that it depends on Spring (as 
Spring framework seem to be the defacto standard for component 
management) and can be setup using standard Spring. If that is not 
possible because the component have some more specialized life stile, we 
need to provide Spring pre or post processors or other mechanisms for 
handling the life style.

Reusable components should also preferably be packaged so that there is 
one API module containing interfaces and classes that a user might want 
to import in application code and one (or several) implementation 
modules containing the implementation of the components together with a 
default Spring configuration file.

My goal for the layering is to be able to build webapps writing sitemaps 
in Java (see 
http://svn.apache.org/repos/asf/cocoon/whiteboard/java-sitemap/src/main/java/org/apache/cocoon/javasitemap/JavaSitemapServlet.java 
for some initial experimentation) and only needing to depend on the 
component management and pipeline layers.

The modules
===========

Currently the core is split into sitemap modules, pipeline modules, 
modules for configuration handling and deployment and some support and 
components modules.

Configuration handling
----------------------

The configuration handling consists of:

  cocoon-configuration-api
  cocoon-spring-configurator

that Carsten have discussed in another thread. The configuration 
handling is responsible for running mode dependent configurations and 
the deployment of resources from blocks.

I'm not certain that the configuration and the deployment stuff belongs 
together. Also to make the deployment util more usable outside Cocoon, 
something like the block context source is needed. Would it be possible 
to implement the block context source as a Spring resource and move it 
to the same module as the deployer?

Pipeline
--------

The pipeline layer consists of:

  cocoon-pipeline-api - Interfaces: ProcessingPipeline, sitemap 
component and basic XML interfaces, the environment abstraction, caching 
interfaces and needed exceptions.
  cocoon-pipeline-impl - The various implementations of 
ProcessingPipeline together with classes that they depend on and various 
abstract classes for the sitemap components.
  cocoon-pipeline-components - generators, transformers, serializers, 
readers, some sources and xpointer and xslt components.

Here probably some of the abstract classes should be moved to the api 
module, but I didn't want to make the api to heavy in the first step. 
Also the impl module probably contains components and utility class that 
would be better to factor out to own modules.

Maybe the component module should be split into a base and an optional 
module?

Right now the dependency graph is:

  api -> impl -> components

while it rather should be

  api -> impl
  api -> components

Sitemap
-------

The sitemap layer consists of:

  cocoon-sitemap-api - Interfaces: Processor, interfaces for actions, 
modules, flow, matching, selection and some support interfaces for 
classloading, notification, processing and the sitemap.
  cocoon-sitemap-impl - the tree processor with dependencies, the spring 
packages for Avalon support, various abstract base classes and support 
classes for the various sitemap components.
  cocoon-sitemap-components - actions, modules, some sources, matchers 
and selectors and the StatusGenerator that presents various sitemap 
specific things.

As for the pipeline modules, the impl module is to heavy, some of the 
abstract base classes and support classes should go to the api module 
other things possibly to own component or support modules. Also here 
maybe the component module should be split into a base and an optional 
module?

The dependency graph looks like:

  sitemap-api  -> sitemap-impl  -> sitemap-components
       ^               ^                ^
       |               |                |
  pipeline-api -> pipeline-impl -> pipeline-components

which (while it looks nice and symmetric) not is that good. ipl and 
components modules should only depend on api and more specific component 
and support modules.

The Avalon support should preferably be moved to a separate module.

XML components
--------------

These components are intended to be reusable outside Cocoon and are 
Spring components.

  xml-api - DOMParser and SAXParser
  xml-impl - implementations with configuration
  xml-resolver - the Cocoon system resolver together with default enteties

Here we could add more generally reusable XML components from the other 
modules after having "deavalonized" them.

Util
----

  cocoon-util - The util package

Here we should try to reduce the number of dependencies further.

Thread components
-----------------

  cocoon-thread-api
  cocoon-thread-impl

The thread components could possibly be replaced with the corresponding 
Spring components 
(http://static.springframework.org/spring/docs/2.0.x/reference/scheduling.html) 
and after that be deprecated.

Store components
----------------

  cocoon-store-impl

Various implementation of the Excalibur Store interface.

Core
----

In the end the core module should only contain the dependencies to all 
the modules that once was part of cocoon-core. Right now it contain 
various components and classes that I don't know where to put or that 
has complicated dependencies so that I cannot put them where it would be 
most natural.

                   --- o0o ---

So as you can see we are starting to get a nice structure, but there are 
still many open questions where your input or even better, work would be 
needed.

/Daniel

Re: Core split status

[Daniel Fagerstrom <da...@nada.kth.se>]

Simone Gianni skrev:
> Daniel Fagerstrom wrote:
>> The pipeline layer consists of:
>>
>>  cocoon-pipeline-api - Interfaces: ProcessingPipeline, sitemap
>> component and basic XML interfaces, the environment abstraction,
>> caching interfaces and needed exceptions.
>>  cocoon-pipeline-impl - The various implementations of
>> ProcessingPipeline together with classes that they depend on and
>> various abstract classes for the sitemap components.
>>  cocoon-pipeline-components - generators, transformers, serializers,
>> readers, some sources and xpointer and xslt components.
>>
>> Here probably some of the abstract classes should be moved to the api
>> module, but I didn't want to make the api to heavy in the first step.
>> Also the impl module probably contains components and utility class
>> that would be better to factor out to own modules.
>>
>> Maybe the component module should be split into a base and an optional
>> module?
>>
>> Right now the dependency graph is:
>>
>>  api -> impl -> components
>>
>> while it rather should be
>>
>>  api -> impl
>>  api -> components
>>
> Hi Daniel, very good work!

Thanks :)

> But I'm missing something : if *-impl
> contains abstract classes for components, and *-components contains that
> components, how can they not depend on the abstract classes they extend?

You are absolutely right. Abstract base classes intended for reuse 
should be part of the API. But as we, IMHO, severely overuse abstract 
base classes in and as the abstract base classes would increase the 
number of dependencies for the API and as we are trying to decrease the 
Avalon dependencies, I felt rather reluctant to put all the abstract 
classes in the API. And because of that the components depends on the 
impl instead of just the API. I hope we will be able to get to the point 
where the abstract base classes have more reasonable dependencies and 
can be moved to that API or maybe some pipeline util module.

/Daniel

Re: Core split status

[Simone Gianni <si...@apache.org>]

Daniel Fagerstrom wrote:
>
> The pipeline layer consists of:
>
>  cocoon-pipeline-api - Interfaces: ProcessingPipeline, sitemap
> component and basic XML interfaces, the environment abstraction,
> caching interfaces and needed exceptions.
>  cocoon-pipeline-impl - The various implementations of
> ProcessingPipeline together with classes that they depend on and
> various abstract classes for the sitemap components.
>  cocoon-pipeline-components - generators, transformers, serializers,
> readers, some sources and xpointer and xslt components.
>
> Here probably some of the abstract classes should be moved to the api
> module, but I didn't want to make the api to heavy in the first step.
> Also the impl module probably contains components and utility class
> that would be better to factor out to own modules.
>
> Maybe the component module should be split into a base and an optional
> module?
>
> Right now the dependency graph is:
>
>  api -> impl -> components
>
> while it rather should be
>
>  api -> impl
>  api -> components
>
Hi Daniel, very good work! But I'm missing something : if *-impl
contains abstract classes for components, and *-components contains that
components, how can they not depend on the abstract classes they extend?

Simone

Re: Core split status

[Daniel Fagerstrom <da...@nada.kth.se>]

Carsten Ziegeler skrev:
> Daniel Fagerstrom wrote:
>> I'm not certain that the configuration and the deployment stuff belongs 
>> together. Also to make the deployment util more usable outside Cocoon, 
>> something like the block context source is needed. Would it be possible 
>> to implement the block context source as a Spring resource and move it 
>> to the same module as the deployer?
> I'm not sure, but I think Spring does not provide a pluggable resource
> resolving system like we have. When you have an application context, the
> application context is the ResourceLoader (= source resolver). So adding
> own protocols means adding functionality to the application context. It
> might be that it's possible to configure an additional resource loader,
> but I haven't looked into that.

I looked at it and AFAICS the only way is to extend the application 
context :/

> The current configurator is doing several things that I think belong
> together. Adding configurations and properties to the Spring application
> context is in fact deploying beans (or configuration files), so I think
> it makes sense to have the deployment of web application resources at
> the same place. If you have a jar with both, bean configurations and a
> COB-INF directory, it might not make sense to deploy only one of those
> twos; they belong together and need to be deployed both.
> 
> I think we should leave this stuff in the configurator for now; release
> this stuff as 1.0 and use it. If the needs arises to split up, we can
> make a configurator 2.0 version and change things.

Makes sense.

>> Pipeline
>> --------
>>
>> </snip/>
>> Maybe the component module should be split into a base and an optional 
>> module?
> Yepp.
> 
>> The Avalon support should preferably be moved to a separate module.
> Yes, I tried it but failed :( Currently the avalon stuff is mixed with
> tree processor stuff, so you can't easily separate one from the other :(
> This has to change!

What are the main remaining problems?

IIUC, one problem is context handling for components. One way to solve 
that could be to create a custom "sitemap scope" for Spring. If the 
components that are directly dependent on what sitemap they are used in 
are defined to be sitemap scoped and use the AOP scoped proxy 
(http://static.springframework.org/spring/docs/2.0.x/reference/beans.html#beans-factory-scopes-custom-using), 
the components that depends on them in turn does not need to know 
anything about the current sitemap context anymore.

/Daniel

Re: Core split status

[Carsten Ziegeler <cz...@apache.org>]

Daniel Fagerstrom wrote:
> I'm not certain that the configuration and the deployment stuff belongs 
> together. Also to make the deployment util more usable outside Cocoon, 
> something like the block context source is needed. Would it be possible 
> to implement the block context source as a Spring resource and move it 
> to the same module as the deployer?
I'm not sure, but I think Spring does not provide a pluggable resource
resolving system like we have. When you have an application context, the
application context is the ResourceLoader (= source resolver). So adding
own protocols means adding functionality to the application context. It
might be that it's possible to configure an additional resource loader,
but I haven't looked into that.

The current configurator is doing several things that I think belong
together. Adding configurations and properties to the Spring application
context is in fact deploying beans (or configuration files), so I think
it makes sense to have the deployment of web application resources at
the same place. If you have a jar with both, bean configurations and a
COB-INF directory, it might not make sense to deploy only one of those
twos; they belong together and need to be deployed both.

I think we should leave this stuff in the configurator for now; release
this stuff as 1.0 and use it. If the needs arises to split up, we can
make a configurator 2.0 version and change things.

> 
> Pipeline
> --------
> 
> </snip/>
> Maybe the component module should be split into a base and an optional 
> module?
Yepp.

> The Avalon support should preferably be moved to a separate module.
Yes, I tried it but failed :( Currently the avalon stuff is mixed with
tree processor stuff, so you can't easily separate one from the other :(
This has to change!

> So as you can see we are starting to get a nice structure, but there are 
> still many open questions where your input or even better, work would be 
> needed.
> 
Amen.

Carsten

-- 
Carsten Ziegeler - Chief Architect
http://www.s-und-n.de
http://www.osoco.org/weblogs/rael/

Re: Core split status

[Reinhard Poetz <re...@apache.org>]

Daniel Fagerstrom wrote:

<snip/>
> So as you can see we are starting to get a nice structure, but there are 
> still many open questions where your input or even better, work would be 
> needed.

thanks for the great work!

-- 
Reinhard Pötz           Independent Consultant, Trainer & (IT)-Coach 

{Software Engineering, Open Source, Web Applications, Apache Cocoon}

                                        web(log): http://www.poetz.cc
--------------------------------------------------------------------