Deployment architecture

[David Jencks <da...@yahoo.com>]

The current architecture of deployment might be considered to have some 
limitations.  It is not clear how to extend the system to deploy more 
artifacts such as web services and portlets, let alone artifacts we 
don't know about yet.  There is also a growing web of dependencies 
between module builders to take care of ejb references, connection 
factory references, and bits such as security and naming.  I also think 
there may be situations where an incompletely processed artifact does 
not cause a deployment error.  I wonder if there is a simpler more 
extensible architecture.

What I'm thinking of has two main new features:

1. a chain of  builders with only one method, "build"
2. some sort of "deferred work" object that allows one builder to ask 
another one, farther down the chain, to do some work for it.

Lets look at how builders interact with each other:

One type of interaction is for a builder to add extract some 
information from the current deployment plan and cache it in the 
deployment context for use by itself or other builders later.  For 
instance, the connector builder figures out the activation spec 
metadata and puts it in the context for use when deploying message 
driven beans.

This type of interaction can be  handled by a chain of builders by 
simply having two builders: the first one adds the shared info to the 
context, and the second one(s) use it.

Another type of interaction is for a builder to extract some elements 
from the spec and/or vendor plan and immediately ask another builder to 
process it.  For instance, ejb and resource references are immediately 
processed by the naming builder, the vendor security descriptor 
elements are immediately processed by the security builder, and 
dependency and gbean elements are processed by the service builder.

Most of these are used to construct a gbean attribute value which is 
not used further during deployment.  So, perhaps the builder needing 
the work done could construct a "deferred work" object containing the 
element to be processed and the gbean and attribute name, and add this 
to the context.  Then the, e.g., naming builder could look in the 
context for deferred work objects that it understands, process them, 
and set the appropriate gbean attribute values. If any deferred work 
objects were left over unprocessed when it came time to serialize the 
gbean state, we would know there was a deployment problem.

The dependency elements in the plans are used to construct the 
classpath, which is needed during deployment. Each builder could 
extract these elements and put them in deferred work objects: in this 
case the processing would not set a gbean attribute value, but would 
result in adding to the classpath.

The ear deployer might fit well into this scheme.  Instead of calling 
specific module builders for each type of module it recognizes, it 
could bundle up the info for each module into a work object that it 
adds to the context: the current module builders would then look in the 
context for the work objects they understand.

Right now I'd characterize these ideas as speculation and wonder if 
anyone else thinks they are worth pursuing further.  I also wonder if 
limiting the objects in the deployer chain to a single method has too 
little structure and giving them several methods to be called in a 
specified order, as with the current architecture, would provide better 
organization without limiting useful functionality.

Many thanks,
david jencks

Re: Deployment architecture

[David Blevins <db...@gluecode.com>]

Just a couple curiosities.

Would moving the coupling between module builders to be coupling on 
work objects achieve much as far as actually decoupling the deployment 
system, or would it just introduce another brittle layer in the 
existing coupling?

Any ideas on how we would dictate the order of the deployment chain?

-David

On Jan 3, 2005, at 3:05 AM, David Jencks wrote:

> The current architecture of deployment might be considered to have 
> some limitations.  It is not clear how to extend the system to deploy 
> more artifacts such as web services and portlets, let alone artifacts 
> we don't know about yet.  There is also a growing web of dependencies 
> between module builders to take care of ejb references, connection 
> factory references, and bits such as security and naming.  I also 
> think there may be situations where an incompletely processed artifact 
> does not cause a deployment error.  I wonder if there is a simpler 
> more extensible architecture.
>
> What I'm thinking of has two main new features:
>
> 1. a chain of  builders with only one method, "build"
> 2. some sort of "deferred work" object that allows one builder to ask 
> another one, farther down the chain, to do some work for it.
>
> Lets look at how builders interact with each other:
>
> One type of interaction is for a builder to add extract some 
> information from the current deployment plan and cache it in the 
> deployment context for use by itself or other builders later.  For 
> instance, the connector builder figures out the activation spec 
> metadata and puts it in the context for use when deploying message 
> driven beans.
>
> This type of interaction can be  handled by a chain of builders by 
> simply having two builders: the first one adds the shared info to the 
> context, and the second one(s) use it.
>
> Another type of interaction is for a builder to extract some elements 
> from the spec and/or vendor plan and immediately ask another builder 
> to process it.  For instance, ejb and resource references are 
> immediately processed by the naming builder, the vendor security 
> descriptor elements are immediately processed by the security builder, 
> and dependency and gbean elements are processed by the service 
> builder.
>
> Most of these are used to construct a gbean attribute value which is 
> not used further during deployment.  So, perhaps the builder needing 
> the work done could construct a "deferred work" object containing the 
> element to be processed and the gbean and attribute name, and add this 
> to the context.  Then the, e.g., naming builder could look in the 
> context for deferred work objects that it understands, process them, 
> and set the appropriate gbean attribute values. If any deferred work 
> objects were left over unprocessed when it came time to serialize the 
> gbean state, we would know there was a deployment problem.
>
> The dependency elements in the plans are used to construct the 
> classpath, which is needed during deployment. Each builder could 
> extract these elements and put them in deferred work objects: in this 
> case the processing would not set a gbean attribute value, but would 
> result in adding to the classpath.
>
> The ear deployer might fit well into this scheme.  Instead of calling 
> specific module builders for each type of module it recognizes, it 
> could bundle up the info for each module into a work object that it 
> adds to the context: the current module builders would then look in 
> the context for the work objects they understand.
>
> Right now I'd characterize these ideas as speculation and wonder if 
> anyone else thinks they are worth pursuing further.  I also wonder if 
> limiting the objects in the deployer chain to a single method has too 
> little structure and giving them several methods to be called in a 
> specified order, as with the current architecture, would provide 
> better organization without limiting useful functionality.
>
> Many thanks,
> david jencks

Re: Deployment architecture

[Dain Sundstrom <ds...@gluecode.com>]

On Jan 3, 2005, at 1:30 PM, David Jencks wrote:

>
> On Jan 3, 2005, at 11:53 AM, Dain Sundstrom wrote:
>
>> On Jan 3, 2005, at 11:04 AM, David Jencks wrote:
>>
>> I'm not sure I like that also.  My thoughts have revolved around 
>> having a single builder responsible for a module type and child 
>> builders that can add stuff the deployment but are not the main 
>> builder.  For example, wars are currently handled by the JettyBuilder 
>> which means that it is this builder that is responsible for coping 
>> the files from the war archive into the deployment context, setting 
>> up the bulk of the class path, and the module GBeans.  Then if we had 
>> a portlet builder, it would get a chance to add to the classpath and 
>> add additional gbeans to the the deployment.  I envision this being 
>> implemented by the web module builder having a collection reference 
>> to child web module builders, and iteration over the set in each 
>> build method.  Is there something I'm missing that makes this 
>> complex?
>
> Well, right now the ear builder has references to a bunch of other 
> builders that it has to explicitly delegate various work to.  Your 
> proposal for the web builder is similar, in that it will have a set of 
> sub-builders that at various times it explicitly calls.

I think you are implying much coupling then I am.  I really mean that 
the child deployers, are just a set of deployers that only (for 
example) see 3 of the 4 deployment phases.  The just get a crack at the 
deployment unit after the main deployer has done its work.

> With an interceptor chain, you wouldn't need these references at all: 
> the ear builder would simply decompose ears into a bunch of modules, 
> and the other builders would find and work on them.  The web builder 
> would just work on the web.xml, and the portlet builder would just add 
> stuff to the context.  One way of looking at this is that with the 
> interceptors we are building a high level domain specific language for 
> deployment, and setting up the interceptor chain is programming 
> deployment in that language.  Instead of java code maintaining and 
> using references from one builder to another, we set up the 
> interceptor order to do the same work for us.   Of course the current 
> reference-based deployment system can also be regarded as a domain 
> specific language.  I think it might be harder to understand since it 
> has many more references and can't be written down linearly.  I'm by 
> no means convinced that my idea is good, but every time I've switched 
> to an interceptor based architecture it has really improved the power 
> and simplicity of the system.

I believe I understand what you are suggesting, I just don't think it 
is simpler.  I actually think this will be much more complex.  When 
deployment breaks, we will have to complex problems, missing 
"down-chain calls" and spurious "down-chain calls".

As a separate issue, I'm a bit skeptical that a chain metaphor will 
work.  My guess it that it is more like a tree, or valve.

>> My gut says, this is way complex and will be hard to debug.  The is 
>> mainly since there is no way to know who is responsible for a piece 
>> of data.  At the end, you have an extra piece of data, and you don't 
>> know if it was optional, if it is bad data, or it was good data and 
>> the expected builder was not online.  Just my gut.
>
> You could well be right, but I'm not sure it is any different from 
> what we have now.

It is definitely different.  The question is is it better? :)

-dain

Deployment Metadata [was: Deployment architecture]

[Dain Sundstrom <ds...@gluecode.com>]

I decided to split the discussion because it is getting too long to 
read :)

On Jan 3, 2005, at 1:30 PM, David Jencks wrote:

> On Jan 3, 2005, at 11:53 AM, Dain Sundstrom wrote:
>
>> IMNSHO this is caused by our usage of XML beans.  When I talked to 
>> the XML beans guys at ApacheCon they told me to put an abstraction 
>> layer between XML beans and our deployment code.  This allows them to 
>> innovate and us to have well defined objects with interfaces.  If we 
>> had interfaces (and normal objects) for our metadata, we would be 
>> able to create generic metadata processors.  I personally believe 
>> that the majority of the "ugliness" of our deployment code is the 
>> lack of abstraction (flexibility) of our metadata objects.
>
> Can you give me some more specific idea of what you are talking about 
> and how it might work?  I'm intrigued but really have no idea what you 
> mean.

I'm going to talk about implementations to make the discussion more 
concrete, but the implementation is not that important.  I think we 
should create a set of classes to represent a deployment unit.  This 
would mean we would have an EAR object which contained modules, which 
themselves are EJB, WEB, Connector or AppClient.  You get the idea.  
The important part is to use interface so a module build can create use 
it's own implementation.  As for how we get the data from XML to an 
Object graph and back, we either write some code that can take an XML 
Beans dom and turn it into a graph, or we use something like Castor or 
JaxB.

Once we have these objects, we can make them do work for us.  For 
example, the EAR object could resolve an ejb-link.  The connector 
module could contain a set of preformed activation spec (well maybe... 
I'm just making that up :)

Follow?

-dain

Re: Deployment architecture

[David Jencks <dj...@gluecode.com>]

On Jan 3, 2005, at 11:53 AM, Dain Sundstrom wrote:

> On Jan 3, 2005, at 11:04 AM, David Jencks wrote:
>
>> Combining a couple of responses...
>> On Jan 3, 2005, at 9:28 AM, David Blevins wrote:
>>
>>> Just a couple curiosities.
>>>
>>> Would moving the coupling between module builders to be coupling on 
>>> work objects achieve much as far as actually decoupling the 
>>> deployment system, or would it just introduce another brittle layer 
>>> in the existing coupling?
>>
>> I don't know:-)  I think the main decoupling effect would be that 
>> each builder would be able to say "someone needs to process this" 
>> rather than "you, in particular, need to process this".  It looks to 
>> me as if the current system is getting more and more classes that 
>> need to know specifically about one another and call specific (often 
>> static) methods on each other.  I'm wondering if there is some way to 
>> make this more uniform and generic and perhaps extensible and in 
>> particular to easily support web service and portlet deployment.
>
> IMNSHO this is caused by our usage of XML beans.  When I talked to the 
> XML beans guys at ApacheCon they told me to put an abstraction layer 
> between XML beans and our deployment code.  This allows them to 
> innovate and us to have well defined objects with interfaces.  If we 
> had interfaces (and normal objects) for our metadata, we would be able 
> to create generic metadata processors.  I personally believe that the 
> majority of the "ugliness" of our deployment code is the lack of 
> abstraction (flexibility) of our metadata objects.

Can you give me some more specific idea of what you are talking about 
and how it might work?  I'm intrigued but really have no idea what you 
mean.

>
>>> Any ideas on how we would dictate the order of the deployment chain?
>>
>> Either writing a little class like the ejb interceptor builders or by 
>> some kind of deployment descriptor.  If each element is a gbean, 
>> linking them into a chain is pretty easy with our current xml based 
>> gbean deployment system.
>
> Why do we need order?
>
>> On Jan 3, 2005, at 9:49 AM, Dain Sundstrom wrote:
>>
>>> I'm a bit confused about how you envision this working, so please 
>>> take my comments with a grain of salt :)
>>
>> Me too, at this point I'm speculating:-)
>>>
>>> On Jan 3, 2005, at 1:05 AM, David Jencks wrote:
>>>
>>>> The current architecture of deployment might be considered to have 
>>>> some limitations.  It is not clear how to extend the system to 
>>>> deploy more artifacts such as web services and portlets, let alone 
>>>> artifacts we don't know about yet.  There is also a growing web of 
>>>> dependencies between module builders to take care of ejb 
>>>> references, connection factory references, and bits such as 
>>>> security and naming.  I also think there may be situations where an 
>>>> incompletely processed artifact does not cause a deployment error.  
>>>> I wonder if there is a simpler more extensible architecture.
>>>>
>>>> What I'm thinking of has two main new features:
>>>>
>>>> 1. a chain of  builders with only one method, "build"
>>>
>>> Do you really mean a chain?  The word chain implies an order to me, 
>>> so is there an order you are thing of?  If not, how about "set" or 
>>> "bag".
>>
>> Definitely an ordered chain of "interceptors".  All deployments would 
>> happen by feeding stuff into the single topmost interceptor.
>
> Why do we need order?  I thin it would help me if you could give an 
> example of what the order would look like.
>
>>> 2. some sort of "deferred work" object that allows one builder to 
>>> ask another one, farther down the chain, to do some work for it.
>>>>
>>>> Lets look at how builders interact with each other:
>>>>
>>>> One type of interaction is for a builder to add extract some 
>>>> information from the current deployment plan and cache it in the 
>>>> deployment context for use by itself or other builders later.  For 
>>>> instance, the connector builder figures out the activation spec 
>>>> metadata and puts it in the context for use when deploying message 
>>>> driven beans.
>>>>
>>>> This type of interaction can be  handled by a chain of builders by 
>>>> simply having two builders: the first one adds the shared info to 
>>>> the context, and the second one(s) use it.
>>>
>>> If I have this correct, you are suggest we switch from a push 
>>> (cache) system, to a pull system.
>> I don't think so, or at least not entirely.  I am suggesting that the 
>> "init context" method would be one interceptor that adds stuff to the 
>> deployment context so it can be used by any later interceptor.  
>> However, instead of say the ejb builder directly calling the naming 
>> builder with snippets of info to build the read-only context, it 
>> would just add an object to the context to be processed later by the 
>> naming builder.  The advantage I see here is that the ejb builder 
>> doesn't need to know anything about who is going to do the work for 
>> it, nor does it need a reference to the naming builder.
>
> If we had normal metadata object, the objects themselves could perform 
> the processing located in our "static" methods.  Anyway, isn't it the 
> job of the EJB builder to provide ejb-refs?  Of course he can delegate 
> it to another object, but doesn't the buck stop with the ejb builder.  
> Said another way, shouldn't that be an internal detail of the ejb 
> builder?
>
>> I think I remember someone mentioning the idea of having some kind of 
>> builder registry so that a builder when faced with some plan bits it 
>> doesn't deal with directly could ask the registry to find the 
>> appropriate builder and process it.  I think this deferred work idea 
>> provides the same functionality without a central registry.
>
> I'm not sure I like that also.  My thoughts have revolved around 
> having a single builder responsible for a module type and child 
> builders that can add stuff the deployment but are not the main 
> builder.  For example, wars are currently handled by the JettyBuilder 
> which means that it is this builder that is responsible for coping the 
> files from the war archive into the deployment context, setting up the 
> bulk of the class path, and the module GBeans.  Then if we had a 
> portlet builder, it would get a chance to add to the classpath and add 
> additional gbeans to the the deployment.  I envision this being 
> implemented by the web module builder having a collection reference to 
> child web module builders, and iteration over the set in each build 
> method.  Is there something I'm missing that makes this complex?

Well, right now the ear builder has references to a bunch of other 
builders that it has to explicitly delegate various work to.  Your 
proposal for the web builder is similar, in that it will have a set of 
sub-builders that at various times it explicitly calls.  With an 
interceptor chain, you wouldn't need these references at all: the ear 
builder would simply decompose ears into a bunch of modules, and the 
other builders would find and work on them.  The web builder would just 
work on the web.xml, and the portlet builder would just add stuff to 
the context.  One way of looking at this is that with the interceptors 
we are building a high level domain specific language for deployment, 
and setting up the interceptor chain is programming deployment in that 
language.  Instead of java code maintaining and using references from 
one builder to another, we set up the interceptor order to do the same 
work for us.   Of course the current reference-based deployment system 
can also be regarded as a domain specific language.  I think it might 
be harder to understand since it has many more references and can't be 
written down linearly.  I'm by no means convinced that my idea is good, 
but every time I've switched to an interceptor based architecture it 
has really improved the power and simplicity of the system.
>
>>> I'm not sure that will work in the case of an EJB ref.  An EJB ref 
>>> has matching rules, that really need holistic view of all EJBs in 
>>> the deployment unit.  Off the top of my head we have the following 
>>> precedence rules:
>>>
>>> * Exact ejb-link specification of EJB module and EJB name
>>> * No module in ejb-link, but an EJB in the current module has the 
>>> same name as the ejb-link name
>>> * No module in ejb-link, but only one EJB in the EAR that has the 
>>> same name as the ejb-link name
>>> * No ejb-link, but only one EJB has the same name as the type and 
>>> interfaces of the ejb-ref
>>>
>>> For most ejb-refs, you need to need to inspect every EJB in the ear.
>>
>> I should be a bit more specific:-)
>>
>> Lets say we're deploying an ear with 2 ejb modules.  Here's a sketch 
>> of what happens:
>>
>> ear-builder: detects modules in the ear and adds 2 ejb-work objects 
>> to context
>>
>> ejb-init-builder: for each ejb-work object, adds name of ejb to 
>> context (i.e., does the init context work from current ejb-builder)
>>
>> (other stuff, like connector deployment...)
>> ejb-builder: for  each ejb in each ejb-work object, starts 
>> constructing the ejb container gbeans.  It adds a naming-work object 
>> to the context that will fill in the read-only-context attribute of 
>> the gbean when it is processed.
>>
>> (other stuff)
>> naming-builder: for each naming-work object in the context, it uses 
>> builds the read-only context and sets the GBeanData attribute value.
>>
>> (other stuff)
>> serializer: checks that there are no unprocessed work objects, and 
>> serializes the configuration
>
> My gut says, this is way complex and will be hard to debug.  The is 
> mainly since there is no way to know who is responsible for a piece of 
> data.  At the end, you have an extra piece of data, and you don't know 
> if it was optional, if it is bad data, or it was good data and the 
> expected builder was not online.  Just my gut.

You could well be right, but I'm not sure it is any different from what 
we have now.
>
>>>> Another type of interaction is for a builder to extract some 
>>>> elements from the spec and/or vendor plan and immediately ask 
>>>> another builder to process it.  For instance, ejb and resource 
>>>> references are immediately processed by the naming builder, the 
>>>> vendor security descriptor elements are immediately processed by 
>>>> the security builder, and dependency and gbean elements are 
>>>> processed by the service builder.
>>>
>>> Why do we need "builders" for security, naming, and gbeans.  Can't 
>>> we get buy with a utility class instead of a full blown service?
>>
>> That's what we have now.  It works, but I'm not thrilled about the 
>> extensive use of static methods.  We also have the client builder 
>> needing a reference to the connector builder.  We want to be able to 
>> deploy connection factories and admin objects (datasources and 
>> topics/queues) from any plan, so at least the ejb, web, and ear 
>> builders would need references to the connector builder as well.  I'm 
>> worried we will get an impenetrable web of connections between 
>> builders that is difficult to set up or maintain, and wondering if it 
>> can be simplified.
>
> I don't like them either.  IMNSHO these static methods belong in the 
> metadata objects, but since we don't have metadata object they end up 
> in helper classes.  I think once you introduce real metadata object, a 
> huge amount of complexity will fall out of the deployment code.

Again, I'd really like to know more about what you are thinking of and 
how it would work.

>
>>>> Most of these are used to construct a gbean attribute value which 
>>>> is not used further during deployment.  So, perhaps the builder 
>>>> needing the work done could construct a "deferred work" object 
>>>> containing the element to be processed and the gbean and attribute 
>>>> name, and add this to the context.  Then the, e.g., naming builder 
>>>> could look in the context for deferred work objects that it 
>>>> understands, process them, and set the appropriate gbean attribute 
>>>> values. If any deferred work objects were left over unprocessed 
>>>> when it came time to serialize the gbean state, we would know there 
>>>> was a deployment problem.
>>>
>>> Ya, you lost me here.  Can you be more specific on where I would use 
>>> "deferred work" and how it would be implemented?
>>
>> Is the ejb example above detailed enough?
>
> Yep.
>
>>>> The dependency elements in the plans are used to construct the 
>>>> classpath, which is needed during deployment. Each builder could 
>>>> extract these elements and put them in deferred work objects: in 
>>>> this case the processing would not set a gbean attribute value, but 
>>>> would result in adding to the classpath.
>>>
>>> Not sure how that is better then what we have today.
>>>
>>>> The ear deployer might fit well into this scheme.  Instead of 
>>>> calling specific module builders for each type of module it 
>>>> recognizes, it could bundle up the info for each module into a work 
>>>> object that it adds to the context: the current module builders 
>>>> would then look in the context for the work objects they 
>>>> understand.
>>>
>>> Again, I'm not sure how that is better, but I think that would be an 
>>> easy change to make.  Instead of having an 4 builders we have a set 
>>> and ask each one if it can handle a specific module (or give it all 
>>> modules and say do what you can).  I'm just now sure it buys us 
>>> much.  I most afraid that we will loose the ability to have good 
>>> error messages since we don't know who is supposed to be responsible 
>>> for a module.
>>
>> I'm proposing that we shift from the model of deployment components 
>> having a list of builders that they iterate over in hopes of finding 
>> one that works, to a model  where you just throw stuff on a single 
>> chain and look to see if everything got done at the end of the chain. 
>>  I don't see how this affects our ability to improve our currently 
>> uninformative error messages: either way, if there's a builder that 
>> recognizes that it should be able to process something, it can 
>> provide a specific error if it fails, and if there isn't, we don't 
>> recognize something so its hard to say what we should have done.
>
> I'm getting the feeling that this is just different, but we have the 
> same problems.

Probably, but with a single well defined end of the chain it might be 
easier to check for unprocessed bits.  I don't see this as being a 
significant argument either way.

thanks
david jencks
>
>>>> Right now I'd characterize these ideas as speculation and wonder if 
>>>> anyone else thinks they are worth pursuing further.  I also wonder 
>>>> if limiting the objects in the deployer chain to a single method 
>>>> has too little structure and giving them several methods to be 
>>>> called in a specified order, as with the current architecture, 
>>>> would provide better organization without limiting useful 
>>>> functionality.
>>>
>>> I did consider using a single "doIt" method when working with 
>>> deployment last time, but the problem is you trade coupling in 
>>> methods (interface) for complexity in the parameters.  For example, 
>>> all of Java could be rewritten with each class having a single 
>>> method doIt(Map data).  I know that you are not implying that we go 
>>> that far, but this demonstrates the complexity trade-off.  I suggest 
>>> something in the middle, a few methods with a few mildly complex 
>>> parameters :)
>>
>> I haven't gotten that far yet:-)  To me the question is whether it's 
>> clearer to put all the, e.g., ejb processing methods in one class and 
>> possibly have lots of builders that do nothing in many of the methods 
>> or have more than one "ejb builder" class, one for each method.  I 
>> suspect that having 2 or 3 methods will be more convenient but I 
>> don't have a firm opinion yet.
>
> I think we should put the responsibility into one class and allow the 
> implementation to choose how to distribute the complexity of that 
> responsibility.
>
> -dain
>

Re: Deployment architecture

[Dain Sundstrom <ds...@gluecode.com>]

On Jan 3, 2005, at 11:04 AM, David Jencks wrote:

> Combining a couple of responses...
> On Jan 3, 2005, at 9:28 AM, David Blevins wrote:
>
>> Just a couple curiosities.
>>
>> Would moving the coupling between module builders to be coupling on 
>> work objects achieve much as far as actually decoupling the 
>> deployment system, or would it just introduce another brittle layer 
>> in the existing coupling?
>
> I don't know:-)  I think the main decoupling effect would be that each 
> builder would be able to say "someone needs to process this" rather 
> than "you, in particular, need to process this".  It looks to me as if 
> the current system is getting more and more classes that need to know 
> specifically about one another and call specific (often static) 
> methods on each other.  I'm wondering if there is some way to make 
> this more uniform and generic and perhaps extensible and in particular 
> to easily support web service and portlet deployment.

IMNSHO this is caused by our usage of XML beans.  When I talked to the 
XML beans guys at ApacheCon they told me to put an abstraction layer 
between XML beans and our deployment code.  This allows them to 
innovate and us to have well defined objects with interfaces.  If we 
had interfaces (and normal objects) for our metadata, we would be able 
to create generic metadata processors.  I personally believe that the 
majority of the "ugliness" of our deployment code is the lack of 
abstraction (flexibility) of our metadata objects.

>> Any ideas on how we would dictate the order of the deployment chain?
>
> Either writing a little class like the ejb interceptor builders or by 
> some kind of deployment descriptor.  If each element is a gbean, 
> linking them into a chain is pretty easy with our current xml based 
> gbean deployment system.

Why do we need order?

> On Jan 3, 2005, at 9:49 AM, Dain Sundstrom wrote:
>
>> I'm a bit confused about how you envision this working, so please 
>> take my comments with a grain of salt :)
>
> Me too, at this point I'm speculating:-)
>>
>> On Jan 3, 2005, at 1:05 AM, David Jencks wrote:
>>
>>> The current architecture of deployment might be considered to have 
>>> some limitations.  It is not clear how to extend the system to 
>>> deploy more artifacts such as web services and portlets, let alone 
>>> artifacts we don't know about yet.  There is also a growing web of 
>>> dependencies between module builders to take care of ejb references, 
>>> connection factory references, and bits such as security and naming. 
>>>  I also think there may be situations where an incompletely 
>>> processed artifact does not cause a deployment error.  I wonder if 
>>> there is a simpler more extensible architecture.
>>>
>>> What I'm thinking of has two main new features:
>>>
>>> 1. a chain of  builders with only one method, "build"
>>
>> Do you really mean a chain?  The word chain implies an order to me, 
>> so is there an order you are thing of?  If not, how about "set" or 
>> "bag".
>
> Definitely an ordered chain of "interceptors".  All deployments would 
> happen by feeding stuff into the single topmost interceptor.

Why do we need order?  I thin it would help me if you could give an 
example of what the order would look like.

>> 2. some sort of "deferred work" object that allows one builder to ask 
>> another one, farther down the chain, to do some work for it.
>>>
>>> Lets look at how builders interact with each other:
>>>
>>> One type of interaction is for a builder to add extract some 
>>> information from the current deployment plan and cache it in the 
>>> deployment context for use by itself or other builders later.  For 
>>> instance, the connector builder figures out the activation spec 
>>> metadata and puts it in the context for use when deploying message 
>>> driven beans.
>>>
>>> This type of interaction can be  handled by a chain of builders by 
>>> simply having two builders: the first one adds the shared info to 
>>> the context, and the second one(s) use it.
>>
>> If I have this correct, you are suggest we switch from a push (cache) 
>> system, to a pull system.
> I don't think so, or at least not entirely.  I am suggesting that the 
> "init context" method would be one interceptor that adds stuff to the 
> deployment context so it can be used by any later interceptor.  
> However, instead of say the ejb builder directly calling the naming 
> builder with snippets of info to build the read-only context, it would 
> just add an object to the context to be processed later by the naming 
> builder.  The advantage I see here is that the ejb builder doesn't 
> need to know anything about who is going to do the work for it, nor 
> does it need a reference to the naming builder.

If we had normal metadata object, the objects themselves could perform 
the processing located in our "static" methods.  Anyway, isn't it the 
job of the EJB builder to provide ejb-refs?  Of course he can delegate 
it to another object, but doesn't the buck stop with the ejb builder.  
Said another way, shouldn't that be an internal detail of the ejb 
builder?

> I think I remember someone mentioning the idea of having some kind of 
> builder registry so that a builder when faced with some plan bits it 
> doesn't deal with directly could ask the registry to find the 
> appropriate builder and process it.  I think this deferred work idea 
> provides the same functionality without a central registry.

I'm not sure I like that also.  My thoughts have revolved around having 
a single builder responsible for a module type and child builders that 
can add stuff the deployment but are not the main builder.  For 
example, wars are currently handled by the JettyBuilder which means 
that it is this builder that is responsible for coping the files from 
the war archive into the deployment context, setting up the bulk of the 
class path, and the module GBeans.  Then if we had a portlet builder, 
it would get a chance to add to the classpath and add additional gbeans 
to the the deployment.  I envision this being implemented by the web 
module builder having a collection reference to child web module 
builders, and iteration over the set in each build method.  Is there 
something I'm missing that makes this complex?

>> I'm not sure that will work in the case of an EJB ref.  An EJB ref 
>> has matching rules, that really need holistic view of all EJBs in the 
>> deployment unit.  Off the top of my head we have the following 
>> precedence rules:
>>
>> * Exact ejb-link specification of EJB module and EJB name
>> * No module in ejb-link, but an EJB in the current module has the 
>> same name as the ejb-link name
>> * No module in ejb-link, but only one EJB in the EAR that has the 
>> same name as the ejb-link name
>> * No ejb-link, but only one EJB has the same name as the type and 
>> interfaces of the ejb-ref
>>
>> For most ejb-refs, you need to need to inspect every EJB in the ear.
>
> I should be a bit more specific:-)
>
> Lets say we're deploying an ear with 2 ejb modules.  Here's a sketch 
> of what happens:
>
> ear-builder: detects modules in the ear and adds 2 ejb-work objects to 
> context
>
> ejb-init-builder: for each ejb-work object, adds name of ejb to 
> context (i.e., does the init context work from current ejb-builder)
>
> (other stuff, like connector deployment...)
> ejb-builder: for  each ejb in each ejb-work object, starts 
> constructing the ejb container gbeans.  It adds a naming-work object 
> to the context that will fill in the read-only-context attribute of 
> the gbean when it is processed.
>
> (other stuff)
> naming-builder: for each naming-work object in the context, it uses 
> builds the read-only context and sets the GBeanData attribute value.
>
> (other stuff)
> serializer: checks that there are no unprocessed work objects, and 
> serializes the configuration

My gut says, this is way complex and will be hard to debug.  The is 
mainly since there is no way to know who is responsible for a piece of 
data.  At the end, you have an extra piece of data, and you don't know 
if it was optional, if it is bad data, or it was good data and the 
expected builder was not online.  Just my gut.

>>> Another type of interaction is for a builder to extract some 
>>> elements from the spec and/or vendor plan and immediately ask 
>>> another builder to process it.  For instance, ejb and resource 
>>> references are immediately processed by the naming builder, the 
>>> vendor security descriptor elements are immediately processed by the 
>>> security builder, and dependency and gbean elements are processed by 
>>> the service builder.
>>
>> Why do we need "builders" for security, naming, and gbeans.  Can't we 
>> get buy with a utility class instead of a full blown service?
>
> That's what we have now.  It works, but I'm not thrilled about the 
> extensive use of static methods.  We also have the client builder 
> needing a reference to the connector builder.  We want to be able to 
> deploy connection factories and admin objects (datasources and 
> topics/queues) from any plan, so at least the ejb, web, and ear 
> builders would need references to the connector builder as well.  I'm 
> worried we will get an impenetrable web of connections between 
> builders that is difficult to set up or maintain, and wondering if it 
> can be simplified.

I don't like them either.  IMNSHO these static methods belong in the 
metadata objects, but since we don't have metadata object they end up 
in helper classes.  I think once you introduce real metadata object, a 
huge amount of complexity will fall out of the deployment code.

>>> Most of these are used to construct a gbean attribute value which is 
>>> not used further during deployment.  So, perhaps the builder needing 
>>> the work done could construct a "deferred work" object containing 
>>> the element to be processed and the gbean and attribute name, and 
>>> add this to the context.  Then the, e.g., naming builder could look 
>>> in the context for deferred work objects that it understands, 
>>> process them, and set the appropriate gbean attribute values. If any 
>>> deferred work objects were left over unprocessed when it came time 
>>> to serialize the gbean state, we would know there was a deployment 
>>> problem.
>>
>> Ya, you lost me here.  Can you be more specific on where I would use 
>> "deferred work" and how it would be implemented?
>
> Is the ejb example above detailed enough?

Yep.

>>> The dependency elements in the plans are used to construct the 
>>> classpath, which is needed during deployment. Each builder could 
>>> extract these elements and put them in deferred work objects: in 
>>> this case the processing would not set a gbean attribute value, but 
>>> would result in adding to the classpath.
>>
>> Not sure how that is better then what we have today.
>>
>>> The ear deployer might fit well into this scheme.  Instead of 
>>> calling specific module builders for each type of module it 
>>> recognizes, it could bundle up the info for each module into a work 
>>> object that it adds to the context: the current module builders 
>>> would then look in the context for the work objects they understand.
>>
>> Again, I'm not sure how that is better, but I think that would be an 
>> easy change to make.  Instead of having an 4 builders we have a set 
>> and ask each one if it can handle a specific module (or give it all 
>> modules and say do what you can).  I'm just now sure it buys us much. 
>>  I most afraid that we will loose the ability to have good error 
>> messages since we don't know who is supposed to be responsible for a 
>> module.
>
> I'm proposing that we shift from the model of deployment components 
> having a list of builders that they iterate over in hopes of finding 
> one that works, to a model  where you just throw stuff on a single 
> chain and look to see if everything got done at the end of the chain.  
> I don't see how this affects our ability to improve our currently 
> uninformative error messages: either way, if there's a builder that 
> recognizes that it should be able to process something, it can provide 
> a specific error if it fails, and if there isn't, we don't recognize 
> something so its hard to say what we should have done.

I'm getting the feeling that this is just different, but we have the 
same problems.

>>> Right now I'd characterize these ideas as speculation and wonder if 
>>> anyone else thinks they are worth pursuing further.  I also wonder 
>>> if limiting the objects in the deployer chain to a single method has 
>>> too little structure and giving them several methods to be called in 
>>> a specified order, as with the current architecture, would provide 
>>> better organization without limiting useful functionality.
>>
>> I did consider using a single "doIt" method when working with 
>> deployment last time, but the problem is you trade coupling in 
>> methods (interface) for complexity in the parameters.  For example, 
>> all of Java could be rewritten with each class having a single method 
>> doIt(Map data).  I know that you are not implying that we go that 
>> far, but this demonstrates the complexity trade-off.  I suggest 
>> something in the middle, a few methods with a few mildly complex 
>> parameters :)
>
> I haven't gotten that far yet:-)  To me the question is whether it's 
> clearer to put all the, e.g., ejb processing methods in one class and 
> possibly have lots of builders that do nothing in many of the methods 
> or have more than one "ejb builder" class, one for each method.  I 
> suspect that having 2 or 3 methods will be more convenient but I don't 
> have a firm opinion yet.

I think we should put the responsibility into one class and allow the 
implementation to choose how to distribute the complexity of that 
responsibility.

-dain

Re: Deployment architecture

[David Blevins <da...@visi.com>]

On Mon, Jan 03, 2005 at 11:04:25AM -0800, David Jencks wrote:
> On Jan 3, 2005, at 9:28 AM, David Blevins wrote:
> 
> >Any ideas on how we would dictate the order of the deployment chain?
> 
> Either writing a little class like the ejb interceptor builders or by 
> some kind of deployment descriptor.  If each element is a gbean, 
> linking them into a chain is pretty easy with our current xml based 
> gbean deployment system.
> >
> >-David
> 
> On Jan 3, 2005, at 9:49 AM, Dain Sundstrom wrote:
>
> >>1. a chain of  builders with only one method, "build"
> >
> >Do you really mean a chain?  The word chain implies an order to me, so 
> >is there an order you are thing of?  If not, how about "set" or "bag".
> 
> Definitely an ordered chain of "interceptors".  All deployments would 
> happen by feeding stuff into the single topmost interceptor.

It seems that if you took our existing deployment code which is very ping-pong-like, e.g. builders bouncing data back and forth from each other in the four phases of deployment, and tried to make it linear, e.g. a chain, you would end up with very macro-level interceptors at the top passing very complex work objects to increasingly micro-level (finer-grained) interceptors toward the bottom which are very dependent on parts of those complex work objects.  

In essence, what you are thinking about is like XSLT stylesheets; a number of overloaded build methods or "templates" processing work objects a tree of "nodes".  In that world you definitely get what i described, a few templates which start out fairly abstract and multiply rapidly into several small templates to process the more specific parts of the tree.

I dunno if that is good or bad.  The clear thing is that we wouldn't end up with a one-phase deployment system, just the four-phases spread out through the chain with strong coupling between producers and consumers of work objects.

Again, still rolling this one around in my head.

-David

Re: Deployment architecture

[David Jencks <dj...@gluecode.com>]

Combining a couple of responses...
On Jan 3, 2005, at 9:28 AM, David Blevins wrote:

> Just a couple curiosities.
>
> Would moving the coupling between module builders to be coupling on 
> work objects achieve much as far as actually decoupling the deployment 
> system, or would it just introduce another brittle layer in the 
> existing coupling?

I don't know:-)  I think the main decoupling effect would be that each 
builder would be able to say "someone needs to process this" rather 
than "you, in particular, need to process this".  It looks to me as if 
the current system is getting more and more classes that need to know 
specifically about one another and call specific (often static) methods 
on each other.  I'm wondering if there is some way to make this more 
uniform and generic and perhaps extensible and in particular to easily 
support web service and portlet deployment.
>
> Any ideas on how we would dictate the order of the deployment chain?

Either writing a little class like the ejb interceptor builders or by 
some kind of deployment descriptor.  If each element is a gbean, 
linking them into a chain is pretty easy with our current xml based 
gbean deployment system.
>
> -David

On Jan 3, 2005, at 9:49 AM, Dain Sundstrom wrote:

> I'm a bit confused about how you envision this working, so please take 
> my comments with a grain of salt :)

Me too, at this point I'm speculating:-)
>
> On Jan 3, 2005, at 1:05 AM, David Jencks wrote:
>
>> The current architecture of deployment might be considered to have 
>> some limitations.  It is not clear how to extend the system to deploy 
>> more artifacts such as web services and portlets, let alone artifacts 
>> we don't know about yet.  There is also a growing web of dependencies 
>> between module builders to take care of ejb references, connection 
>> factory references, and bits such as security and naming.  I also 
>> think there may be situations where an incompletely processed 
>> artifact does not cause a deployment error.  I wonder if there is a 
>> simpler more extensible architecture.
>>
>> What I'm thinking of has two main new features:
>>
>> 1. a chain of  builders with only one method, "build"
>
> Do you really mean a chain?  The word chain implies an order to me, so 
> is there an order you are thing of?  If not, how about "set" or "bag".

Definitely an ordered chain of "interceptors".  All deployments would 
happen by feeding stuff into the single topmost interceptor.
>
>> 2. some sort of "deferred work" object that allows one builder to ask 
>> another one, farther down the chain, to do some work for it.
>>
>> Lets look at how builders interact with each other:
>>
>> One type of interaction is for a builder to add extract some 
>> information from the current deployment plan and cache it in the 
>> deployment context for use by itself or other builders later.  For 
>> instance, the connector builder figures out the activation spec 
>> metadata and puts it in the context for use when deploying message 
>> driven beans.
>>
>> This type of interaction can be  handled by a chain of builders by 
>> simply having two builders: the first one adds the shared info to the 
>> context, and the second one(s) use it.
>
> If I have this correct, you are suggest we switch from a push (cache) 
> system, to a pull system.
I don't think so, or at least not entirely.  I am suggesting that the 
"init context" method would be one interceptor that adds stuff to the 
deployment context so it can be used by any later interceptor.  
However, instead of say the ejb builder directly calling the naming 
builder with snippets of info to build the read-only context, it would 
just add an object to the context to be processed later by the naming 
builder.  The advantage I see here is that the ejb builder doesn't need 
to know anything about who is going to do the work for it, nor does it 
need a reference to the naming builder.

I think I remember someone mentioning the idea of having some kind of 
builder registry so that a builder when faced with some plan bits it 
doesn't deal with directly could ask the registry to find the 
appropriate builder and process it.  I think this deferred work idea 
provides the same functionality without a central registry.
>
> I'm not sure that will work in the case of an EJB ref.  An EJB ref has 
> matching rules, that really need holistic view of all EJBs in the 
> deployment unit.  Off the top of my head we have the following 
> precedence rules:
>
> * Exact ejb-link specification of EJB module and EJB name
> * No module in ejb-link, but an EJB in the current module has the same 
> name as the ejb-link name
> * No module in ejb-link, but only one EJB in the EAR that has the same 
> name as the ejb-link name
> * No ejb-link, but only one EJB has the same name as the type and 
> interfaces of the ejb-ref
>
> For most ejb-refs, you need to need to inspect every EJB in the ear.

I should be a bit more specific:-)

Lets say we're deploying an ear with 2 ejb modules.  Here's a sketch of 
what happens:

ear-builder: detects modules in the ear and adds 2 ejb-work objects to 
context

ejb-init-builder: for each ejb-work object, adds name of ejb to context 
(i.e., does the init context work from current ejb-builder)

(other stuff, like connector deployment...)
ejb-builder: for  each ejb in each ejb-work object, starts constructing 
the ejb container gbeans.  It adds a naming-work object to the context 
that will fill in the read-only-context attribute of the gbean when it 
is processed.

(other stuff)
naming-builder: for each naming-work object in the context, it uses 
builds the read-only context and sets the GBeanData attribute value.

(other stuff)
serializer: checks that there are no unprocessed work objects, and 
serializes the configuration.
>
>> Another type of interaction is for a builder to extract some elements 
>> from the spec and/or vendor plan and immediately ask another builder 
>> to process it.  For instance, ejb and resource references are 
>> immediately processed by the naming builder, the vendor security 
>> descriptor elements are immediately processed by the security 
>> builder, and dependency and gbean elements are processed by the 
>> service builder.
>
> Why do we need "builders" for security, naming, and gbeans.  Can't we 
> get buy with a utility class instead of a full blown service?

That's what we have now.  It works, but I'm not thrilled about the 
extensive use of static methods.  We also have the client builder 
needing a reference to the connector builder.  We want to be able to 
deploy connection factories and admin objects (datasources and 
topics/queues) from any plan, so at least the ejb, web, and ear 
builders would need references to the connector builder as well.  I'm 
worried we will get an impenetrable web of connections between builders 
that is difficult to set up or maintain, and wondering if it can be 
simplified.
>
>> Most of these are used to construct a gbean attribute value which is 
>> not used further during deployment.  So, perhaps the builder needing 
>> the work done could construct a "deferred work" object containing the 
>> element to be processed and the gbean and attribute name, and add 
>> this to the context.  Then the, e.g., naming builder could look in 
>> the context for deferred work objects that it understands, process 
>> them, and set the appropriate gbean attribute values. If any deferred 
>> work objects were left over unprocessed when it came time to 
>> serialize the gbean state, we would know there was a deployment 
>> problem.
>
> Ya, you lost me here.  Can you be more specific on where I would use 
> "deferred work" and how it would be implemented?

Is the ejb example above detailed enough?
>
>> The dependency elements in the plans are used to construct the 
>> classpath, which is needed during deployment. Each builder could 
>> extract these elements and put them in deferred work objects: in this 
>> case the processing would not set a gbean attribute value, but would 
>> result in adding to the classpath.
>
> Not sure how that is better then what we have today.
>
>> The ear deployer might fit well into this scheme.  Instead of calling 
>> specific module builders for each type of module it recognizes, it 
>> could bundle up the info for each module into a work object that it 
>> adds to the context: the current module builders would then look in 
>> the context for the work objects they understand.
>
> Again, I'm not sure how that is better, but I think that would be an 
> easy change to make.  Instead of having an 4 builders we have a set 
> and ask each one if it can handle a specific module (or give it all 
> modules and say do what you can).  I'm just now sure it buys us much.  
> I most afraid that we will loose the ability to have good error 
> messages since we don't know who is supposed to be responsible for a 
> module.

I'm proposing that we shift from the model of deployment components 
having a list of builders that they iterate over in hopes of finding 
one that works, to a model  where you just throw stuff on a single 
chain and look to see if everything got done at the end of the chain.  
I don't see how this affects our ability to improve our currently 
uninformative error messages: either way, if there's a builder that 
recognizes that it should be able to process something, it can provide 
a specific error if it fails, and if there isn't, we don't recognize 
something so its hard to say what we should have done.
>
>> Right now I'd characterize these ideas as speculation and wonder if 
>> anyone else thinks they are worth pursuing further.  I also wonder if 
>> limiting the objects in the deployer chain to a single method has too 
>> little structure and giving them several methods to be called in a 
>> specified order, as with the current architecture, would provide 
>> better organization without limiting useful functionality.
>
> I did consider using a single "doIt" method when working with 
> deployment last time, but the problem is you trade coupling in methods 
> (interface) for complexity in the parameters.  For example, all of 
> Java could be rewritten with each class having a single method 
> doIt(Map data).  I know that you are not implying that we go that far, 
> but this demonstrates the complexity trade-off.  I suggest something 
> in the middle, a few methods with a few mildly complex parameters :)

I haven't gotten that far yet:-)  To me the question is whether it's 
clearer to put all the, e.g., ejb processing methods in one class and 
possibly have lots of builders that do nothing in many of the methods 
or have more than one "ejb builder" class, one for each method.  I 
suspect that having 2 or 3 methods will be more convenient but I don't 
have a firm opinion yet.

Many thanks,
david jencks

>
> -dain
>

Re: Deployment architecture

[Dain Sundstrom <ds...@gluecode.com>]

I'm a bit confused about how you envision this working, so please take 
my comments with a grain of salt :)

On Jan 3, 2005, at 1:05 AM, David Jencks wrote:

> The current architecture of deployment might be considered to have 
> some limitations.  It is not clear how to extend the system to deploy 
> more artifacts such as web services and portlets, let alone artifacts 
> we don't know about yet.  There is also a growing web of dependencies 
> between module builders to take care of ejb references, connection 
> factory references, and bits such as security and naming.  I also 
> think there may be situations where an incompletely processed artifact 
> does not cause a deployment error.  I wonder if there is a simpler 
> more extensible architecture.
>
> What I'm thinking of has two main new features:
>
> 1. a chain of  builders with only one method, "build"

Do you really mean a chain?  The word chain implies an order to me, so 
is there an order you are thing of?  If not, how about "set" or "bag".

> 2. some sort of "deferred work" object that allows one builder to ask 
> another one, farther down the chain, to do some work for it.
>
> Lets look at how builders interact with each other:
>
> One type of interaction is for a builder to add extract some 
> information from the current deployment plan and cache it in the 
> deployment context for use by itself or other builders later.  For 
> instance, the connector builder figures out the activation spec 
> metadata and puts it in the context for use when deploying message 
> driven beans.
>
> This type of interaction can be  handled by a chain of builders by 
> simply having two builders: the first one adds the shared info to the 
> context, and the second one(s) use it.

If I have this correct, you are suggest we switch from a push (cache) 
system, to a pull system.

I'm not sure that will work in the case of an EJB ref.  An EJB ref has 
matching rules, that really need holistic view of all EJBs in the 
deployment unit.  Off the top of my head we have the following 
precedence rules:

* Exact ejb-link specification of EJB module and EJB name
* No module in ejb-link, but an EJB in the current module has the same 
name as the ejb-link name
* No module in ejb-link, but only one EJB in the EAR that has the same 
name as the ejb-link name
* No ejb-link, but only one EJB has the same name as the type and 
interfaces of the ejb-ref

For most ejb-refs, you need to need to inspect every EJB in the ear.

> Another type of interaction is for a builder to extract some elements 
> from the spec and/or vendor plan and immediately ask another builder 
> to process it.  For instance, ejb and resource references are 
> immediately processed by the naming builder, the vendor security 
> descriptor elements are immediately processed by the security builder, 
> and dependency and gbean elements are processed by the service 
> builder.

Why do we need "builders" for security, naming, and gbeans.  Can't we 
get buy with a utility class instead of a full blown service?

> Most of these are used to construct a gbean attribute value which is 
> not used further during deployment.  So, perhaps the builder needing 
> the work done could construct a "deferred work" object containing the 
> element to be processed and the gbean and attribute name, and add this 
> to the context.  Then the, e.g., naming builder could look in the 
> context for deferred work objects that it understands, process them, 
> and set the appropriate gbean attribute values. If any deferred work 
> objects were left over unprocessed when it came time to serialize the 
> gbean state, we would know there was a deployment problem.

Ya, you lost me here.  Can you be more specific on where I would use 
"deferred work" and how it would be implemented?

> The dependency elements in the plans are used to construct the 
> classpath, which is needed during deployment. Each builder could 
> extract these elements and put them in deferred work objects: in this 
> case the processing would not set a gbean attribute value, but would 
> result in adding to the classpath.

Not sure how that is better then what we have today.

> The ear deployer might fit well into this scheme.  Instead of calling 
> specific module builders for each type of module it recognizes, it 
> could bundle up the info for each module into a work object that it 
> adds to the context: the current module builders would then look in 
> the context for the work objects they understand.

Again, I'm not sure how that is better, but I think that would be an 
easy change to make.  Instead of having an 4 builders we have a set and 
ask each one if it can handle a specific module (or give it all modules 
and say do what you can).  I'm just now sure it buys us much.  I most 
afraid that we will loose the ability to have good error messages since 
we don't know who is supposed to be responsible for a module.

> Right now I'd characterize these ideas as speculation and wonder if 
> anyone else thinks they are worth pursuing further.  I also wonder if 
> limiting the objects in the deployer chain to a single method has too 
> little structure and giving them several methods to be called in a 
> specified order, as with the current architecture, would provide 
> better organization without limiting useful functionality.

I did consider using a single "doIt" method when working with 
deployment last time, but the problem is you trade coupling in methods 
(interface) for complexity in the parameters.  For example, all of Java 
could be rewritten with each class having a single method doIt(Map 
data).  I know that you are not implying that we go that far, but this 
demonstrates the complexity trade-off.  I suggest something in the 
middle, a few methods with a few mildly complex parameters :)

-dain