You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avalon.apache.org by Peter Donald <pe...@apache.org> on 2002/06/26 16:16:19 UTC
[RT] One Container to Rule them all
Hi,
MetaInfo
--------
Lets assume we can come up with a central container. First thing we need to
do is to is externalize metadata for all the components. This is painful to
maintain so we will need to use XDoclet to annotate our code. These
annotations would essentially state the resources that the component
requires and the resources that the component is capable of providing. A
fully specced out java file would look something like the following. Note
that is is vastly more annotated than a real example so most files wont be
this complex.
/**
* @avalon.component name="mail-server-impl" version="1.2.4"
lifecycle="thread-safe"
* @avalon.service type="org.apache.avalon.MailServer"
*/
public class MailServerImpl
implements MailServer, LogEnabled, Contextualizable, Composable
{
/**
* @avalon.logger description="Base logger passed in is used for ...
logging stuff"
* @avalon.logger name="auth"
* description="Auth sub category is used to log
authentication information"
*/
public void enableLogging( Logger l )
{
m_logger = l;
m_authLogger = l.getChildLogger( "auth" );
}
/**
* @avalon.context type="org.apache.phoenix.api.BlockContext"
* @avalon.entry key="mBeanServer" type="javax.management.MBeanServer"
optional="false"
*/
public void contextualize( Context c )
{
m_blockContext = (BlockContext)c;
MBeanServer mbs = m_blockContext.get( "mBeanServer" );
doMBeanRegister( mbs );
}
/**
* @avalon.dependency type="org.apache.cornerstone.SomeServiceInterface"
* @avalon.dependency role="quirky" type="org.apache.SomeInterface"
optional="true"
*/
public void compose( ComponentManager cm )
{
m_someServiceInterface = (SomeServiceInterface)cm.lookup(
SomeServiceInterface.ROLE );
if( cm.exists( "quirky" ) )
{
m_someInterface = (SomeInterface)cm.lookup( "quirky" );
}
}
}
Of course the above is massively complex but it demonstrates the possible
annotations that could exist. The metainfo system that is currently under
development also allows extensible set of attributes to be associated with
various resources. So in theory if you needed container specific metadata
you could associate it with different features to achieve that extension.
For example, if cocoon had a transformer X that only transformed documents
that conformed to the "http://xml.apache.org/cocoon/announcement.dtd" then
you could annotate the class to indicate this. If it also spat out another
DTD you could add anotations for this via something like
/**
* @avalon.component
cocoon:input-dtd="http://xml.apache.org/cocoon/announcement.dtd"
* @avalon.component
cocoon:output-dtd="http://xml.apache.org/cocoon/other.dtd"
*/
class XTransformer implements Transformer { ... }
The cocooon container could then be extended to specially deal with such
attributes. Cocoon could verify that the input/output is chained correctly
and that whole sitemap once assembled is valid.
With enough annotations you could almost validate the entire sitemap prior
to deploying it which would probably save a lot of headaches over time.
So the first part of our strategy to moving towards a single container is
creating a generic MetaInfo infrastructure.
Component/Assembly Profile
--------------------------
Where MetaInfo gives information about the type of a component, the Profile
describes information about a component in an Application/assembly. So it
saids that Component A has a dependency on Component B and has
configuration X. A set of Component Profiles make up an Assembly Profile.
So usually a Profile is made up of something like
Component A of Type P, uses Component B and C, and has Configuration X
Component B of Type Q, uses Component C, and has Parameters Y
Component C of Type R, uses no Components, and has Configuration Z
The actual arrangement is partially container specific but the general form
is common.
It is the Profile that gives the container the ability to validate the
application before instantiating it. ie You can make sure that all the
dependencies are valid, configuration is valid according to a schema etc.
So after the Profile is validated the application should startup in 99% of
cases except when runtime errors occur.
Component Entrys
----------------
When a container is managing a component it creates an Entry per component.
The Entry manages all the runtime information associated with a component.
This runtime information is container specific. If the component is pooled
then the Entry will contain a pool of instances. If the container tracks
resources, the Entry will contain list of resources the component uses. If
the component is accessed via proxies, the entry will list the proxies.
The Process of Assembly
-----------------------
The process of assembly is creating an application profile via wiring
together components.
In the past some containers, such as Phoenix, went the path of requiring
assembler to explicitly specify all the components and resolve all the
dependencies. ie If a component has a dependency then the assembler must
specify another component which will satisfy the dependency. So you have to
manually assemble the application. This can be a bit error prone especially
when you need to take into consideration such aspects as thread-safety,
inability to have recursive dependencies and potentially many many many
components.
Other containers may auto-assemble the application. For example I believe
Merlin does something like the following (though I may not be 100% accurate
it is close enough). When you start an "application" you declare a
component that is not fully resolved. Merlin then kicks in and tries to
assemble a full Profile. For every unsatisfied dependency in scans the
Profiles of all the Components and sees if there is any candidates that can
satisfy dependency. If there is one candidate that satisfies dependency
then it is used. If there is multiple candidates that satisfy dependency
then a heuristic is employed to select one of the candidates.
The heuristic is currently governed by a combination of things I believe.
It has a policy attribute in MetaInfo of dependecyDescriptor that it can
use, it also makes sure that no circular dependencies are created and in
reality the evaluation process could include oodles more variables. So lets
generalize it to the following interface
public interface DependencyEvaluator
{
int evaluate( ComponentMetaData consumer,
DependencyDescriptor dependency,
ComponentMetaData candidate );
}
Each candidate is passed through evaluator and a score collected. The
candidate with highest score "wins" and is selected as provider for
dependency. Anyways after walking the components, Merlin eventually builds
up a full application Profile. Using this mechanism the assembly requires
far less work by assembler as the runtime will make educated guesses on the
best possible dependency for anything that is not fully specified.
In reality Fortress is in a similar situation except that its mapping is
more structured and does not currently follow metadata (as no such metadata
exists).
Handlers
--------
Each component may have what we call a different "lifestyle". ie Components
may be single-client, singl-threaded, pooled, "singleton" etc. For each of
these different lifestyles we are going to need a slightly different
architecture via which to aquire the component.
The component may still be passed through standard lifecycle process and
described by standard Profile and standard MetaInfo but it will have a
different handler. The handler will enforce the different lifestyle.
ie If the component is sharable it will hand out the same instance to
multiple consumers. If the component is not sharable between multiple
consumers then it will hand out different instances to different consumers.
The handlers is one of the main places where the containers will differ.
Some will offer pooling and advanced resource management. Others will proxy
access to components, maybe offering interceptors etc.
Implementation Overview
-----------------------
So how do we go about implementing this?
First of all the basic MetaInfo structure can be shared between containers
with ease. No container should extend these classes (in fact they are final
so they cant). Container specific attributes can be stored for each
different feature. There is currently no standard for these extra
attributes. Eventually it may be prudent to adopt some "standard"
attributes but now it is mostly free form that we can use to experiment
with stuff.
The Component/Application Profile classes provide a basis for all the
Profile information. However it is possible that some containers will
extend these classes to provide specific information relevent only to the
particular container. However for many containers (ie Phoenix and Fortress)
the base Profile classes should be sufficient.
Almost every container will have different implementations for
ComponentEntry and the different ComponentHandlers. The implementation of
these features effectively define how the container works.
Shared Container Parts
----------------------
There is significant overlap in the code for writing the container. So how
do we go about sharing it all?
* All containers can share the metainfo code (from containerkit)
* All containers can share the lifecycle processing code (from containerkit)
* Dependency traversal can be shared by all containers (from containerkit)
* Merlin and Fortress should definetly share the "auto-assembly" utility
classes.
* Phoenix and Merlin can share the Handler/ComponentEntry part of container
Theres the possibility that we may be able to share some of the other bits
but thats something we can think about later.
Benefits of all this?
---------------------
The biggest benefit of all this is that we will finally have the ability to
write components and transparently deploy them into other containers with
very little effort. It is likely that there will still be some container
specific jazz in some components but we can get at least 90% cross
container compatability.
So that means Myrmidon will be able to use Cocoon services (yea!), Phoenix
will be able to use Merlin services and all the other combinations.
Containers will then be differentiated by their;
* features (Pooling, auto assembly, isolation, multi-app support,
hierarchial support)
* resource cost (how long to startup, memory usage etc)
* deployment format. (ie Phoenix has .sars while other containers use the
ClassLoader they are already in).
* assembly descriptors (ie how you specify that components are wired
together. Compare sitemap vs the assembly.xml, config.xml, environment.xml
format of Phoenix vs single file for myrmidon)
Conclusion
----------
It is not perfect, we have not got no grail but it is as close as we are
going to get right now. We specify just enough that we can achieve 90%
component portability between containers but we still leave room for
different containers being customized for different things and specializing
in different areas.
Thoughts?
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [RT] One Container to Rule them all
Posted by Peter Royal <pr...@apache.org>.
On Thursday 27 June 2002 03:42 am, Leo Simons wrote:
> On Thu, 2002-06-27 at 09:11, Peter Donald wrote:
> > At 08:57 AM 6/27/2002 +0200, you wrote:
> > > > With enough annotations you could almost validate the entire sitemap
> > > > prior to deploying it which would probably save a lot of headaches
> > > > over time.
> > >
> > >is the current metainfo material you guys are writing startup-only? It
> > >has been said cocoon's sitemap changes at runtime
> >
> > And I would put it to you that it doesn't. The other Leo has said much
> > the same thing.
>
> hmm. It seems to me that it would be a valid use case to have the
> Assembly Profile change at runtime, and not the metainfo itself. ie:
> drop in a new component at runtime (say through jmx or a commandline
> tool like the axis package has), and the profile changes.
>
> The separation between assembly and metainfo is still a little clouded
> in my head, so...
On the "changing at runtime" bit wrt Cocoon. What changes is the sitemap, but
the sitemap is its own container.
If you modify the sitemap, the existing container is discarded and a new one
is created. Thus it is still at container startup.
However, dynamically adding components would be huge. I know this is something
Jason Van Zyl is looking for/towards.
-pete
--
peter royal -> proyal@apache.org
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [RT] One Container to Rule them all
Posted by Peter Donald <pe...@apache.org>.
At 12:33 PM 6/27/2002 +0200, you wrote:
> > hmm. It seems to me that it would be a valid use case to have the
> > Assembly Profile change at runtime, and not the metainfo itself. ie:
> > drop in a new component at runtime (say through jmx or a commandline
> > tool like the axis package has), and the profile changes.
>
> >From what I understand, component metadata is there to stay
hopefully.
>and to be
>somehow embedded/hardwired into the component behavior (using
>XDoclet-generated code wrapping, proxying (that would be slower),
>specific bytecode extensions of whatever comes next). Is this assumption
>correct?
I am not sure what you mean. Currently we just generator XML descriptors
and the container reads descriptor and interprets the ComponentInfo as
appropraite. We could generate wrapping code but not sure where it would be
used exactly.
Cheers,
Peter Donald
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Faced with the choice between changing one's mind,
and proving that there is no need to do so - almost
everyone gets busy on the proof."
- John Kenneth Galbraith
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [RT] One Container to Rule them all
Posted by Stefano Mazzocchi <st...@apache.org>.
Leo Simons wrote:
>
> On Thu, 2002-06-27 at 09:11, Peter Donald wrote:
> > At 08:57 AM 6/27/2002 +0200, you wrote:
> > > > With enough annotations you could almost validate the entire sitemap prior
> > > > to deploying it which would probably save a lot of headaches over time.
> > >
> > >is the current metainfo material you guys are writing startup-only? It
> > >has been said cocoon's sitemap changes at runtime
> >
> > And I would put it to you that it doesn't. The other Leo has said much the
> > same thing.
The sitemap may change at runtime, but anytime it changes, it restarts.
So, from a component assembly point of view, it doesn't change.
For example, we were careful *not* to allow the pipelines to be
constructed with run-time information. They are just choosen with that
information, but all the pipelines are assembled and prepared before the
request comes.
Then, for productivity issues, we made it possible for the sitemap to
restart when the sitemap.xmap files change on disk. This is a feature
for development, we suggest to turn it off on production.
> hmm. It seems to me that it would be a valid use case to have the
> Assembly Profile change at runtime, and not the metainfo itself. ie:
> drop in a new component at runtime (say through jmx or a commandline
> tool like the axis package has), and the profile changes.
>>From what I understand, component metadata is there to stay and to be
somehow embedded/hardwired into the component behavior (using
XDoclet-generated code wrapping, proxying (that would be slower),
specific bytecode extensions of whatever comes next). Is this assumption
correct?
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [RT] One Container to Rule them all
Posted by Leo Simons <le...@apache.org>.
On Thu, 2002-06-27 at 09:11, Peter Donald wrote:
> At 08:57 AM 6/27/2002 +0200, you wrote:
> > > With enough annotations you could almost validate the entire sitemap prior
> > > to deploying it which would probably save a lot of headaches over time.
> >
> >is the current metainfo material you guys are writing startup-only? It
> >has been said cocoon's sitemap changes at runtime
>
> And I would put it to you that it doesn't. The other Leo has said much the
> same thing.
hmm. It seems to me that it would be a valid use case to have the
Assembly Profile change at runtime, and not the metainfo itself. ie:
drop in a new component at runtime (say through jmx or a commandline
tool like the axis package has), and the profile changes.
The separation between assembly and metainfo is still a little clouded
in my head, so...
> > Component Entrys
> > > ----------------
> > >
> > > When a container is managing a component it creates an Entry per
> > component.
> > > The Entry manages all the runtime information associated with a component.
> >
> >not the Assembly Profile?
>
> maybe a reference to Profile
hmm. If you see an assembly as possibly changing at runtime, that means
"runtime information" includes a "runtime assembly profile"....maybe in
the next refactoring... ;)
> > > The handlers is one of the main places where the containers will differ.
> > > Some will offer pooling and advanced resource management. Others will
> > proxy
> > > access to components, maybe offering interceptors etc.
> >
> >what is the minimum? ie what should the-other-leos MicroContainer
> >support?
>
> See SimpleServiceKernel for an example of basic container. Does little bar
> run things.
will do. Where is it?
> >do we make it a specification of the standard Profile classes that they
> >are sufficient for Phoenix/Fortress (or do we specify phoenix/fortress
> >don't support more than the standard Profile). I'd like that.
>
> It is just how they happen to stand at this stage of evolution.
> Fortres//Phoenix are free to extend them in future if needed.
okay. You're still against formalizing container feature specification
then?
thx,
- LSD
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [RT] One Container to Rule them all
Posted by Peter Donald <pe...@apache.org>.
At 08:57 AM 6/27/2002 +0200, you wrote:
> > With enough annotations you could almost validate the entire sitemap prior
> > to deploying it which would probably save a lot of headaches over time.
>
>is the current metainfo material you guys are writing startup-only? It
>has been said cocoon's sitemap changes at runtime
And I would put it to you that it doesn't. The other Leo has said much the
same thing.
> Component Entrys
> > ----------------
> >
> > When a container is managing a component it creates an Entry per
> component.
> > The Entry manages all the runtime information associated with a component.
>
>not the Assembly Profile?
maybe a reference to Profile
> > The handlers is one of the main places where the containers will differ.
> > Some will offer pooling and advanced resource management. Others will
> proxy
> > access to components, maybe offering interceptors etc.
>
>what is the minimum? ie what should the-other-leos MicroContainer
>support?
See SimpleServiceKernel for an example of basic container. Does little bar
run things.
> The Component/Application Profile classes provide a basis for all the
> > Profile information. However it is possible that some containers will
> > extend these classes to provide specific information relevent only to the
> > particular container. However for many containers (ie Phoenix and
> Fortress)
> > the base Profile classes should be sufficient.
>
>do we make it a specification of the standard Profile classes that they
>are sufficient for Phoenix/Fortress (or do we specify phoenix/fortress
>don't support more than the standard Profile). I'd like that.
It is just how they happen to stand at this stage of evolution.
Fortres//Phoenix are free to extend them in future if needed.
>seems to me the basic definition of Entry
>
>public interface Entry
>{
> ComponentMetaData getMetaData();
> RuntimeContext getRuntimeContext();
> Handler getHandler();
>}
>
>or something similar could be common to many containers (ie
>Handler/ComponentEntry stuff can partially be in containerkit)
Similar in concept but implementations vary. Making all container implement
a heavy weight interface like the above would never fly.
Cheers,
Peter Donald
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Faced with the choice between changing one's mind,
and proving that there is no need to do so - almost
everyone gets busy on the proof."
- John Kenneth Galbraith
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [RT] One Container to Rule them all
Posted by Peter Royal <pr...@apache.org>.
On Thursday 27 June 2002 06:40 am, Stefano Mazzocchi wrote:
> Counterexamples:
>
> 1) FileGenerator, where the schema of the output is bound to the
> resource being parsed and it's available at runtime only.
But what if there was something like:
<map:generate type="file" src="cocoon://myfile.xml">
<map:output-schema src="cocoon://myfile.xsd"/>
</map:generate>
> 2) XSLTTransformer, where the transforming instructions (thus the schema
> constraints) are bound to the stylesheets and not the code that
> interprets it. (Things change with XSLTC which is able to compile
> stylesheets, but XSLTC will never be Avalon-aware anyway).
And:
<map:transform type="xslt" src="cocoon://mytransform.xsl">
<map:input-schema src="cocoon://mytransform-input.xsd"/>
<map:output-schema src="cocoon://mytransform-output.xsd"/>
</map:transform>
Upon further dwelling, doing this at the pipeline level would probably be
overkill, but for the blocks you have been working on, it might make sense at
the block entry/exit points.
> Still, I have the feeling that Cocoon should not use the same container
> to handle sitemap components and the other more general components.
I agree. Cocoon's sitemap is a special case and will require a specialized
container.
-pete
--
peter royal -> proyal@apache.org
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [RT] One Container to Rule them all
Posted by Stefano Mazzocchi <st...@apache.org>.
Peter Donald wrote:
> For example, if cocoon had a transformer X that only transformed documents
> that conformed to the "http://xml.apache.org/cocoon/announcement.dtd" then
> you could annotate the class to indicate this. If it also spat out another
> DTD you could add anotations for this via something like
>
> /**
> * @avalon.component
> cocoon:input-dtd="http://xml.apache.org/cocoon/announcement.dtd"
> * @avalon.component
> cocoon:output-dtd="http://xml.apache.org/cocoon/other.dtd"
> */
> class XTransformer implements Transformer { ... }
>
> The cocooon container could then be extended to specially deal with such
> attributes. Cocoon could verify that the input/output is chained correctly
> and that whole sitemap once assembled is valid.
>
> With enough annotations you could almost validate the entire sitemap prior
> to deploying it which would probably save a lot of headaches over time.
Works on paper, but not in real life.
Counterexamples:
1) FileGenerator, where the schema of the output is bound to the
resource being parsed and it's available at runtime only.
2) XSLTTransformer, where the transforming instructions (thus the schema
constraints) are bound to the stylesheets and not the code that
interprets it. (Things change with XSLTC which is able to compile
stylesheets, but XSLTC will never be Avalon-aware anyway).
Given than the above two are by far the most used sitemap components, I
wouldn't use the above example as a feature of the new assembly
capabilities.
- o -
Besides this, I like what I see and I like the goal of being able to
reuse Cocoon components (not the sitemap ones, those aren't useful
anywhere else) in other containers.
Still, I have the feeling that Cocoon should not use the same container
to handle sitemap components and the other more general components.
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [RT] One Container to Rule them all
Posted by Stephen McConnell <mc...@osm.net>.
Peter Donald wrote:
> At 04:37 AM 6/28/2002 +0200, you wrote:
>
>>> The Component/Application Profile classes provide a basis for all
>>> the Profile information. However it is possible that some containers
>>> will extend these classes to provide specific information relevent
>>> only to the particular container. However for many containers (ie
>>> Phoenix and Fortress) the base Profile classes should be sufficient.
>>
>>
>>
>> :-)
>> Pete - are you hinting that Merlin may want more?
>
>
> Yep - so will myrmidon, most likely.
>
>>> There is significant overlap in the code for writing the container.
>>> So how do we go about sharing it all?
>>>
>>> * All containers can share the metainfo code (from containerkit)
>>> * All containers can share the lifecycle processing code (from
>>> containerkit)
>>
>>
>>
>> One problem with the lifecycle processing code. Currently the
>> codebase abstracts the processing down to startup and shutdown and a
>> per component basis. In Merlin I'm using parrallel componet
>> lifecycle processing which means that components aquire references to
>> each other before initialization, and that during the start pahse, a
>> component knows that all of its dependent service providers have
>> already been initialized. I guess all I want to do at this stage is
>> flag the potential for alternative lifecyle handlers.
>
>
> ...
>
>>> * Dependency traversal can be shared by all containers (from
>>> containerkit)
>>
>>
>> One of the things I came up with in the above parrallel processing
>> approach was the difference between a dependecy on a service
>> reference as opposed to a service usage dependecy. Service reference
>> dependecies should not restrict depedecy validation - however - using
>> the current dependecy declaration we don't have a way of saying that
>> the dependecy is only on the service reference.
>
>
> Dont get what you are saying. All providers are initialized before
> consumers aquire them and there should never be a reference to an
> object before it is good to go. So what is it you are saying exactly.
What I am saying is that there is a distinct difference between a
reference dependecy as opposed to an usage depedency. I have several
components that do not use services aquired though dependecy
declarations until some event occurs while the dependent component is
servicing client requests. In these cases the component has a
"reference" dependency - and this case enables elimination of the
recursive dependency trap - providing the container knows that the
dependent component declares that it will not use the reference until
post initialization. Given this knowlege, you can eliminate most
recusive depedency problems. As things stand - yes - we need to ensure
that the service is associated with a fully prepared component - but
that's only because we have no notion of the difference between a
reference to a dependecy as opposed to a usage depedency.
Cheers, Steve.
--
Stephen J. McConnell
OSM SARL
digital products for a global economy
mailto:mcconnell@osm.net
http://www.osm.net
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [RT] One Container to Rule them all
Posted by Peter Donald <pe...@apache.org>.
At 04:37 AM 6/28/2002 +0200, you wrote:
>>The Component/Application Profile classes provide a basis for all the
>>Profile information. However it is possible that some containers will
>>extend these classes to provide specific information relevent only to the
>>particular container. However for many containers (ie Phoenix and
>>Fortress) the base Profile classes should be sufficient.
>
>
>:-)
>Pete - are you hinting that Merlin may want more?
Yep - so will myrmidon, most likely.
>>There is significant overlap in the code for writing the container. So
>>how do we go about sharing it all?
>>
>>* All containers can share the metainfo code (from containerkit)
>>* All containers can share the lifecycle processing code (from containerkit)
>
>
>One problem with the lifecycle processing code. Currently the codebase
>abstracts the processing down to startup and shutdown and a per component
>basis. In Merlin I'm using parrallel componet lifecycle processing which
>means that components aquire references to each other before
>initialization, and that during the start pahse, a component knows that
>all of its dependent service providers have already been initialized. I
>guess all I want to do at this stage is flag the potential for alternative
>lifecyle handlers.
...
>>* Dependency traversal can be shared by all containers (from containerkit)
>
>One of the things I came up with in the above parrallel processing
>approach was the difference between a dependecy on a service reference as
>opposed to a service usage dependecy. Service reference dependecies
>should not restrict depedecy validation - however - using the current
>dependecy declaration we don't have a way of saying that the dependecy is
>only on the service reference.
Dont get what you are saying. All providers are initialized before
consumers aquire them and there should never be a reference to an object
before it is good to go. So what is it you are saying exactly.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [RT] One Container to Rule them all
Posted by Stephen McConnell <mc...@osm.net>.
Peter Donald wrote:
> Hi,
>
> MetaInfo
> --------
>
> Lets assume we can come up with a central container. First thing we
> need to do is to is externalize metadata for all the components. This
> is painful to maintain so we will need to use XDoclet to annotate our
> code. These annotations would essentially state the resources that the
> component requires and the resources that the component is capable of
> providing. A fully specced out java file would look something like the
> following. Note that is is vastly more annotated than a real example
> so most files wont be this complex.
>
> /**
> * @avalon.component name="mail-server-impl" version="1.2.4"
> lifecycle="thread-safe"
> * @avalon.service type="org.apache.avalon.MailServer"
> */
> public class MailServerImpl
> implements MailServer, LogEnabled, Contextualizable, Composable
> {
> /**
> * @avalon.logger description="Base logger passed in is used for ...
> logging stuff"
> * @avalon.logger name="auth"
> * description="Auth sub category is used to log
> authentication information"
> */
> public void enableLogging( Logger l )
> {
> m_logger = l;
> m_authLogger = l.getChildLogger( "auth" );
> }
>
> /**
> * @avalon.context type="org.apache.phoenix.api.BlockContext"
> * @avalon.entry key="mBeanServer"
> type="javax.management.MBeanServer" optional="false"
> */
> public void contextualize( Context c )
> {
> m_blockContext = (BlockContext)c;
> MBeanServer mbs = m_blockContext.get( "mBeanServer" );
> doMBeanRegister( mbs );
> }
>
> /**
> * @avalon.dependency
> type="org.apache.cornerstone.SomeServiceInterface"
> * @avalon.dependency role="quirky" type="org.apache.SomeInterface"
> optional="true"
> */
> public void compose( ComponentManager cm )
> {
> m_someServiceInterface = (SomeServiceInterface)cm.lookup(
> SomeServiceInterface.ROLE );
> if( cm.exists( "quirky" ) )
> {
> m_someInterface = (SomeInterface)cm.lookup( "quirky" );
> }
> }
> }
>
> Of course the above is massively complex but it demonstrates the
> possible annotations that could exist. The metainfo system that is
> currently under development also allows extensible set of attributes
> to be associated with various resources. So in theory if you needed
> container specific metadata you could associate it with different
> features to achieve that extension.
As part of containerkit doc we need to put in place (a) some formal tag
documentation, and (b) the standard translation of tags in terms of
generated metainfo. If you want to put the basics in place I can start
to fill in details.
>
>
> For example, if cocoon had a transformer X that only transformed
> documents that conformed to the
> "http://xml.apache.org/cocoon/announcement.dtd" then you could
> annotate the class to indicate this. If it also spat out another DTD
> you could add anotations for this via something like
>
> /**
> * @avalon.component
> cocoon:input-dtd="http://xml.apache.org/cocoon/announcement.dtd"
> * @avalon.component
> cocoon:output-dtd="http://xml.apache.org/cocoon/other.dtd"
> */
> class XTransformer implements Transformer { ... }
>
> The cocooon container could then be extended to specially deal with
> such attributes. Cocoon could verify that the input/output is chained
> correctly and that whole sitemap once assembled is valid.
>
> With enough annotations you could almost validate the entire sitemap
> prior to deploying it which would probably save a lot of headaches
> over time.
>
> So the first part of our strategy to moving towards a single container
> is creating a generic MetaInfo infrastructure.
>
> Component/Assembly Profile
> --------------------------
>
> Where MetaInfo gives information about the type of a component, the
> Profile describes information about a component in an
> Application/assembly. So it saids that Component A has a dependency on
> Component B and has configuration X. A set of Component Profiles make
> up an Assembly Profile.
>
> So usually a Profile is made up of something like
>
> Component A of Type P, uses Component B and C, and has Configuration X
> Component B of Type Q, uses Component C, and has Parameters Y
> Component C of Type R, uses no Components, and has Configuration Z
>
> The actual arrangement is partially container specific but the general
> form is common.
>
> It is the Profile that gives the container the ability to validate the
> application before instantiating it. ie You can make sure that all the
> dependencies are valid, configuration is valid according to a schema etc.
>
> So after the Profile is validated the application should startup in
> 99% of cases except when runtime errors occur.
>
> Component Entrys
> ----------------
>
> When a container is managing a component it creates an Entry per
> component. The Entry manages all the runtime information associated
> with a component. This runtime information is container specific. If
> the component is pooled then the Entry will contain a pool of
> instances. If the container tracks resources, the Entry will contain
> list of resources the component uses. If the component is accessed via
> proxies, the entry will list the proxies.
>
>
> The Process of Assembly
> -----------------------
>
> The process of assembly is creating an application profile via wiring
> together components.
>
> In the past some containers, such as Phoenix, went the path of
> requiring assembler to explicitly specify all the components and
> resolve all the dependencies. ie If a component has a dependency then
> the assembler must specify another component which will satisfy the
> dependency. So you have to manually assemble the application. This can
> be a bit error prone especially when you need to take into
> consideration such aspects as thread-safety, inability to have
> recursive dependencies and potentially many many many components.
>
> Other containers may auto-assemble the application. For example I
> believe Merlin does something like the following (though I may not be
> 100% accurate it is close enough). When you start an "application" you
> declare a component that is not fully resolved. Merlin then kicks in
> and tries to assemble a full Profile. For every unsatisfied dependency
> in scans the Profiles of all the Components and sees if there is any
> candidates that can satisfy dependency. If there is one candidate that
> satisfies dependency then it is used. If there is multiple candidates
> that satisfy dependency then a heuristic is employed to select one of
> the candidates.
Correct.
>
> The heuristic is currently governed by a combination of things I
> believe. It has a policy attribute in MetaInfo of dependecyDescriptor
> that it can use, it also makes sure that no circular dependencies are
> created and in reality the evaluation process could include oodles
> more variables. So lets generalize it to the following interface
>
> public interface DependencyEvaluator
> {
> int evaluate( ComponentMetaData consumer,
> DependencyDescriptor dependency,
> ComponentMetaData candidate );
> }
>
> Each candidate is passed through evaluator and a score collected. The
> candidate with highest score "wins" and is selected as provider for
> dependency. Anyways after walking the components, Merlin eventually
> builds up a full application Profile. Using this mechanism the
> assembly requires far less work by assembler as the runtime will make
> educated guesses on the best possible dependency for anything that is
> not fully specified.
This is what I'm working on at the moment.
>
> In reality Fortress is in a similar situation except that its mapping
> is more structured and does not currently follow metadata (as no such
> metadata exists).
>
> Handlers
> --------
>
> Each component may have what we call a different "lifestyle". ie
> Components may be single-client, singl-threaded, pooled, "singleton"
> etc. For each of these different lifestyles we are going to need a
> slightly different architecture via which to aquire the component.
>
> The component may still be passed through standard lifecycle process
> and described by standard Profile and standard MetaInfo but it will
> have a different handler. The handler will enforce the different
> lifestyle.
>
> ie If the component is sharable it will hand out the same instance to
> multiple consumers. If the component is not sharable between multiple
> consumers then it will hand out different instances to different
> consumers.
>
> The handlers is one of the main places where the containers will
> differ. Some will offer pooling and advanced resource management.
> Others will proxy access to components, maybe offering interceptors etc.
From what I have seen so far - Phoenix, Merlin, Fortress and ECM use
some form of handler. Currently the pooled handler support in Merlin is
disabled - planning on bring this back based on Fortress/mpool stuff -
one of the things here is really getting a grip on the Fortress
relationship needed (if any, or if I can do this based on mpool alone).
>
>
> Implementation Overview
> -----------------------
>
> So how do we go about implementing this?
>
> First of all the basic MetaInfo structure can be shared between
> containers with ease. No container should extend these classes (in
> fact they are final so they cant). Container specific attributes can
> be stored for each different feature. There is currently no standard
> for these extra attributes. Eventually it may be prudent to adopt some
> "standard" attributes but now it is mostly free form that we can use
> to experiment with stuff.
Irrespective of the "standard" attribute question - we still need to
document attributes and preferrably in one place. Getting a list of
attributes that are used/needed and cross referenced to which
container(s) is/are using the attribute will give us a bettewr feel for
what experiments are going on and will help to ensure that we avaiod
different names for the same things.
>
> The Component/Application Profile classes provide a basis for all the
> Profile information. However it is possible that some containers will
> extend these classes to provide specific information relevent only to
> the particular container. However for many containers (ie Phoenix and
> Fortress) the base Profile classes should be sufficient.
:-)
Pete - are you hinting that Merlin may want more?
In practice I think the base profile will be sufficient - the only case
I see for additional information concerning the inclusion of context
value declarations (which is only required when a component declares a
context type and context values that are required). Without this any
component that declares a context type as required becomes compoent
specific (e.g. a component declaring a context type of BlockContext
becomes Phoenix specific if the profile does not contain context
creation criteria).
>
> Almost every container will have different implementations for
> ComponentEntry and the different ComponentHandlers. The implementation
> of these features effectively define how the container works.
>
> Shared Container Parts
> ----------------------
>
> There is significant overlap in the code for writing the container. So
> how do we go about sharing it all?
>
> * All containers can share the metainfo code (from containerkit)
> * All containers can share the lifecycle processing code (from
> containerkit)
One problem with the lifecycle processing code. Currently the codebase
abstracts the processing down to startup and shutdown and a per
component basis. In Merlin I'm using parrallel componet lifecycle
processing which means that components aquire references to each other
before initialization, and that during the start pahse, a component
knows that all of its dependent service providers have already been
initialized. I guess all I want to do at this stage is flag the
potential for alternative lifecyle handlers.
>
> * Dependency traversal can be shared by all containers (from
> containerkit)
One of the things I came up with in the above parrallel processing
approach was the difference between a dependecy on a service reference
as opposed to a service usage dependecy. Service reference dependecies
should not restrict depedecy validation - however - using the current
dependecy declaration we don't have a way of saying that the dependecy
is only on the service reference.
>
> * Merlin and Fortress should definetly share the "auto-assembly"
> utility classes.
Actually - the auto-assembly approach could also be used to generate an
inital/default Phonix assembly.xml file.
>
> * Phoenix and Merlin can share the Handler/ComponentEntry part of
> container
>
> Theres the possibility that we may be able to share some of the other
> bits but thats something we can think about later.
>
> Benefits of all this?
> ---------------------
>
> The biggest benefit of all this is that we will finally have the
> ability to write components and transparently deploy them into other
> containers with very little effort. It is likely that there will still
> be some container specific jazz in some components but we can get at
> least 90% cross container compatability.
>
> So that means Myrmidon will be able to use Cocoon services (yea!),
> Phoenix will be able to use Merlin services and all the other
> combinations.
>
> Containers will then be differentiated by their;
> * features (Pooling, auto assembly, isolation, multi-app support,
> hierarchial support)
> * resource cost (how long to startup, memory usage etc)
> * deployment format. (ie Phoenix has .sars while other containers use
> the ClassLoader they are already in).
> * assembly descriptors (ie how you specify that components are wired
> together. Compare sitemap vs the assembly.xml, config.xml,
> environment.xml format of Phoenix vs single file for myrmidon)
>
> Conclusion
> ----------
>
> It is not perfect, we have not got no grail but it is as close as we
> are going to get right now. We specify just enough that we can achieve
> 90% component portability between containers but we still leave room
> for different containers being customized for different things and
> specializing in different areas.
>
> Thoughts?
Curent development using containerkit is going smoothly and the recent
addition of a end-to-end runnable solution is a lot of help is figuring
things out. I'm close to being able to provide an assembly profile
which I'll be interesting in plugging into a variable to the default
kernel to see what I can break.
;-)
Cheers, Steve.
--
Stephen J. McConnell
OSM SARL
digital products for a global economy
mailto:mcconnell@osm.net
http://www.osm.net
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: [RT] One Container to Rule them all
Posted by Leo Simons <le...@apache.org>.
> MetaInfo
> --------
>
> Lets assume we can come up with a central container. First thing we need to
> do is to is externalize metadata for all the components. This is painful to
> maintain so we will need to use XDoclet to annotate our code.
I'm all for presenting XDoclet to maintain metainfo as an option as
opposed to the only way. The current setup where metainfo files are
generated from javadoc is nice; as you have the option of using some
other tool (say a rich wizard-based GUI ;) instead.
> With enough annotations you could almost validate the entire sitemap prior
> to deploying it which would probably save a lot of headaches over time.
is the current metainfo material you guys are writing startup-only? It
has been said cocoon's sitemap changes at runtime so it'd be best if the
metainfo were decoupled from the stage an application is in.
> So the first part of our strategy to moving towards a single container is
> creating a generic MetaInfo infrastructure.
yup.
> Component/Assembly Profile
> --------------------------
Assembly Profile I like, "Component Profile" I find a very confusing
term.
> Component Entrys
> ----------------
>
> When a container is managing a component it creates an Entry per component.
> The Entry manages all the runtime information associated with a component.
not the Assembly Profile?
> The heuristic is currently governed by a combination of things I believe.
> It has a policy attribute in MetaInfo of dependecyDescriptor that it can
> use, it also makes sure that no circular dependencies are created and in
> reality the evaluation process could include oodles more variables. So lets
> generalize it to the following interface
>
> public interface DependencyEvaluator
> {
> int evaluate( ComponentMetaData consumer,
> DependencyDescriptor dependency,
> ComponentMetaData candidate );
> }
generalize pattern == cool.
generalize interface/formalize ?= are we ready yet?
> Handlers
> --------
>
> Each component may have what we call a different "lifestyle". ie Components
> may be single-client, singl-threaded, pooled, "singleton" etc. For each of
> these different lifestyles we are going to need a slightly different
> architecture via which to aquire the component.
we also need to define these lifestyles as part of the framework
metainfo contracts. Otherwise reuse goes out the window again. I'm all
for specifying a few more than would be necessary in brilliant COP if it
means framework users won't have to extend.
> The component may still be passed through standard lifecycle process and
> described by standard Profile and standard MetaInfo but it will have a
> different handler. The handler will enforce the different lifestyle.
seems to me that the mapping we do is
Container --- use to manage Component ---> Handler
|---- create based on metadata -----^
(lifestyle is metadata). Mapping between handler class and component
would probably be best if based on ComponentMetaData instead of on just
"lifestyle".
> The handlers is one of the main places where the containers will differ.
> Some will offer pooling and advanced resource management. Others will proxy
> access to components, maybe offering interceptors etc.
what is the minimum? ie what should the-other-leos MicroContainer
support?
> The Component/Application Profile classes provide a basis for all the
> Profile information. However it is possible that some containers will
> extend these classes to provide specific information relevent only to the
> particular container. However for many containers (ie Phoenix and Fortress)
> the base Profile classes should be sufficient.
do we make it a specification of the standard Profile classes that they
are sufficient for Phoenix/Fortress (or do we specify phoenix/fortress
don't support more than the standard Profile). I'd like that.
> Shared Container Parts
> ----------------------
>
> There is significant overlap in the code for writing the container. So how
> do we go about sharing it all?
>
> * All containers can share the metainfo code (from containerkit)
> * All containers can share the lifecycle processing code (from containerkit)
> * Dependency traversal can be shared by all containers (from containerkit)
> * Merlin and Fortress should definetly share the "auto-assembly" utility
> classes.
> * Phoenix and Merlin can share the Handler/ComponentEntry part of container
seems to me the basic definition of Entry
public interface Entry
{
ComponentMetaData getMetaData();
RuntimeContext getRuntimeContext();
Handler getHandler();
}
or something similar could be common to many containers (ie
Handler/ComponentEntry stuff can partially be in containerkit)
> Theres the possibility that we may be able to share some of the other bits
> but thats something we can think about later.
'kay.
> Benefits of all this?
> ---------------------
>
> The biggest benefit of all this is that we will finally have the ability to
> write components and transparently deploy them into other containers with
> very little effort. It is likely that there will still be some container
> specific jazz in some components but we can get at least 90% cross
> container compatability.
>
> So that means Myrmidon will be able to use Cocoon services (yea!), Phoenix
> will be able to use Merlin services and all the other combinations.
and then what happens to these services and components? We can either
put them all in one place, or just make a reference to all of them in
one place. Need to do that though. No reuse if you don't know there is
something to reuse.
My feeling is it would be coolest if they'd all be in commons (or our
own commons, ie excalibur, if that remains problematic).
> It is not perfect, we have not got no grail but it is as close as we are
> going to get right now. We specify just enough that we can achieve 90%
> component portability between containers but we still leave room for
> different containers being customized for different things and specializing
> in different areas.
>
> Thoughts?
Generally, I like. Specifically, well, some points as noted above and
I'm sure there'll be more when I dive into actual code ;)
grz,
- Leo
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>