Re: [picocontainer-dev] Re: ClassLoader Architecture

[Rickard Öberg <ri...@senselogic.se>]

Paul Hammant wrote:
> Comp A and B (in separate classloaders) can see Common-API, which can 
> see the classloader that contains PicoContainer who's parent is the 
> classes in rt.jar
<snip>

One thing that I have whined about before, and I'm not sure whether it's 
been fixed yet, is that a component A needs to be able to have an 
internal structure (component!=1 object) and it needs to be able to 
expose API implementations to other components without having to expose 
its own internal structure. In the discussions so far I have not seen 
mention of this problem (but maybe I have simply missed it).

I have my own solution for fixing this, but I am wondering if a more 
standard solution to this is available? (now or later) My own solution 
is more of a best-practice use of Pico rather than an extension, and if 
anyone is interested I can describe it in more detail.

It really is a big big big issue once you get either loads of components 
or large components. We have both cases in our system right now.

/Rickard

Re: [picocontainer-dev] Re: ClassLoader Architecture

[Dain Sundstrom <da...@iq80.com>]

Rickard,

It's good to see you around here :)

It is going to take me a few days to digest this, especially  
considering I'm at JavaOne right now, but I was curious if you had  
looked at the OSGi spec.   I think they have captured most of these  
ideas in the spec, but as Paul points out they do it in a very  
"dependency lookup" style.  If you have read the spec, I'm curious  
what you think of their model in comparison to what you have shown here.

-dain

On Jun 26, 2005, at 11:04 PM, Rickard Öberg wrote:

> Jeremy Boynes wrote:
>
>> I would be very interested if you have the time.
>>
>
> Since you asked :-)
>
> There are a couple of issues I wanted to fix with such a scheme.
>
> First of all, a component should be able to expose an API, and only  
> that API should be exposed by the component. Other components  
> should not be able to access helper objects or components, and they  
> should not be able to cast the component to other interfaces than  
> what is explicitly exposed.
>
> Second, a component must be able to either use other API's exposed  
> by other components (trivial) *or* have an internal structure where  
> it hosts components as helpers, and which are not exposed to other  
> components. An example of the latter might be a DataSource that is  
> specific for a component and which should not be autowired into any  
> other component. A component should not "leak" by accident.
>
> Third, components must be lazy-loadable, yet eager-resolvable. Our  
> system is so large that starting up all components at startup-time  
> takes way too long. Components (or at least some) should be lazy- 
> loadable(/lazy-startable) upon the first call to any of the methods  
> in the exposed API.
>
> The basic approach is to use ContainerComposers (CC), which (IIRC)  
> is how MCA works as well. One component, one CC. I then have a base  
> class that allows me to do things like this:
> public class PortletContainerComponent
>    extends ComponentContainerComposer
> {
>    public void composeContainer(MutablePicoContainer parent, Object o)
>    {
>       MutablePicoContainer container = makeChildContainer(parent);
>
>       container.registerComponentImplementation 
> (PortletRegistryImpl.class);
>       container.registerComponentImplementation 
> (PortletDeployer.class);
> container.registerComponentImplementation 
> (TomcatJMXPortletDeployer.class);
>       container.registerComponentImplementation 
> (PortletRenderer.class);
>
>       register(PortletRegistry.class);
>       register(PortletRenderer.class);
>    }
> }
> ---
> A component using this approach always does three things:
> 1) set up a child container for the internal structure. Using  
> makeChildContainer() on the provided parent is the most common, but  
> it can be more complex if you want to.
> 2) configure the internal structure
> 3) expose the API, which will register the API in "parent"
>
> Requirement 1 and 3 above makes it necessary to introduce proxies  
> which can be eagerly resolved by other components without having to  
> have an actual backend component at the resolution time. This, of  
> course, also allows components to be passivated at any time if  
> necessary, but so far the lazy-start is the most important to me.  
> By using the implementation hiding component adapter it is also  
> ensured that only the API is exposed. For example, if the exposed  
> class PortletRegistryImpl implements Startable this is not visible  
> to "parent" above. Hence lifecycle events will only be executed in  
> the internal container. Creating and registering the proxy is done  
> in step 3) above.
>
> To allow for explicit lazy-loading of components I can do things  
> like this:
> public class SearchEngineComponent
>    extends ComponentContainerComposer
> {
>    public void composeContainer(MutablePicoContainer con, Object o)
>    {
>       MutablePicoContainer container = makeChildContainer(con,  
> LAZY_LOAD);
>       container.registerComponentImplementation 
> (SearchEngineImpl.class);
> container.registerComponentImplementation 
> (ClientSearchEngineImpl.class);
>
>       register(SearchEngine.class);
>       register(SearchIndex.class);
>       register(ClientSearch.class);
>    }
> }
> ---
> This will ensure that the objects in this component will only be  
> instantiated and started iff a method of the exposed API is called.  
> Lazy-loading can only be done if none of the objects start threads  
> or exposes some static methods that are not exposed through a Pico  
> API interface. For example, the above would be only half of the  
> search component in our system, with the indexing (having threads)  
> added like so:
> public class IndexingComponent
>    extends ComponentContainerComposer
> {
>    public void composeContainer(MutablePicoContainer con, Object o)
>    {
>       MutablePicoContainer container = makeChildContainer(con);
> container.registerComponentImplementation(SearchIndexOptimizer.class);
>       container.registerComponentImplementation 
> (IndexMaintainer.class);
> container.registerComponentImplementation 
> (IndexMaintainListener.class);
>       container.registerComponentImplementation(IndexRebuilder.class);
>
>       register(IndexRebuilder.class);
>       register(IndexMaintainer.class);
>    }
> }
> ---
> ... since IndexMaintainer uses threads and hence needs to be  
> started before anyone calls its API. These two together implement  
> the "search component" in our system:
> public class SearchComponent
>    extends ComponentContainerComposer
> {
>    public void composeContainer(MutablePicoContainer con, Object o)
>    {
>       new IndexingComponent().composeContainer(con, o);
>       new SearchEngineComponent().composeContainer(con, o);
>    }
> }
> ---
> Note that in this case the composer does not create a sub- 
> container, but instead composes itself directly using the other  
> composers. If it had created a new child container it would have to  
> explicitly register() the API of the child components in order to  
> "push" them upwards. This explicitness ensures that it is  
> impossible to accidentally access the internal structure of a  
> component.
>
> That's about it I think. Some implementation details in  
> ComponentContainerComposer, and there's no class loading going on  
> above (would be easy to add it though).
>
> Overall, this scheme should make it possible to have lots of  
> components, large components, composed components, expose what you  
> want, hide what you want, and yet be reasonably easy to understand  
> and use.
>
> Comments, thoughts, suggestions?
>
> /Rickard
>

Re: [picocontainer-dev] Re: ClassLoader Architecture

[Rickard Öberg <ri...@senselogic.se>]

Jeremy Boynes wrote:
> I would be very interested if you have the time.

Since you asked :-)

There are a couple of issues I wanted to fix with such a scheme.

First of all, a component should be able to expose an API, and only that 
API should be exposed by the component. Other components should not be 
able to access helper objects or components, and they should not be able 
to cast the component to other interfaces than what is explicitly exposed.

Second, a component must be able to either use other API's exposed by 
other components (trivial) *or* have an internal structure where it 
hosts components as helpers, and which are not exposed to other 
components. An example of the latter might be a DataSource that is 
specific for a component and which should not be autowired into any 
other component. A component should not "leak" by accident.

Third, components must be lazy-loadable, yet eager-resolvable. Our 
system is so large that starting up all components at startup-time takes 
way too long. Components (or at least some) should be 
lazy-loadable(/lazy-startable) upon the first call to any of the methods 
in the exposed API.

The basic approach is to use ContainerComposers (CC), which (IIRC) is 
how MCA works as well. One component, one CC. I then have a base class 
that allows me to do things like this:
public class PortletContainerComponent
    extends ComponentContainerComposer
{
    public void composeContainer(MutablePicoContainer parent, Object o)
    {
       MutablePicoContainer container = makeChildContainer(parent);

       container.registerComponentImplementation(PortletRegistryImpl.class);
       container.registerComponentImplementation(PortletDeployer.class);
 
container.registerComponentImplementation(TomcatJMXPortletDeployer.class);
       container.registerComponentImplementation(PortletRenderer.class);

       register(PortletRegistry.class);
       register(PortletRenderer.class);
    }
}
---
A component using this approach always does three things:
1) set up a child container for the internal structure. Using 
makeChildContainer() on the provided parent is the most common, but it 
can be more complex if you want to.
2) configure the internal structure
3) expose the API, which will register the API in "parent"

Requirement 1 and 3 above makes it necessary to introduce proxies which 
can be eagerly resolved by other components without having to have an 
actual backend component at the resolution time. This, of course, also 
allows components to be passivated at any time if necessary, but so far 
the lazy-start is the most important to me. By using the implementation 
hiding component adapter it is also ensured that only the API is 
exposed. For example, if the exposed class PortletRegistryImpl 
implements Startable this is not visible to "parent" above. Hence 
lifecycle events will only be executed in the internal container. 
Creating and registering the proxy is done in step 3) above.

To allow for explicit lazy-loading of components I can do things like this:
public class SearchEngineComponent
    extends ComponentContainerComposer
{
    public void composeContainer(MutablePicoContainer con, Object o)
    {
       MutablePicoContainer container = makeChildContainer(con, LAZY_LOAD);
       container.registerComponentImplementation(SearchEngineImpl.class);
 
container.registerComponentImplementation(ClientSearchEngineImpl.class);

       register(SearchEngine.class);
       register(SearchIndex.class);
       register(ClientSearch.class);
    }
}
---
This will ensure that the objects in this component will only be 
instantiated and started iff a method of the exposed API is called. 
Lazy-loading can only be done if none of the objects start threads or 
exposes some static methods that are not exposed through a Pico API 
interface. For example, the above would be only half of the search 
component in our system, with the indexing (having threads) added like so:
public class IndexingComponent
    extends ComponentContainerComposer
{
    public void composeContainer(MutablePicoContainer con, Object o)
    {
       MutablePicoContainer container = makeChildContainer(con);
 
container.registerComponentImplementation(SearchIndexOptimizer.class);
       container.registerComponentImplementation(IndexMaintainer.class);
 
container.registerComponentImplementation(IndexMaintainListener.class);
       container.registerComponentImplementation(IndexRebuilder.class);

       register(IndexRebuilder.class);
       register(IndexMaintainer.class);
    }
}
---
... since IndexMaintainer uses threads and hence needs to be started 
before anyone calls its API. These two together implement the "search 
component" in our system:
public class SearchComponent
    extends ComponentContainerComposer
{
    public void composeContainer(MutablePicoContainer con, Object o)
    {
       new IndexingComponent().composeContainer(con, o);
       new SearchEngineComponent().composeContainer(con, o);
    }
}
---
Note that in this case the composer does not create a sub-container, but 
instead composes itself directly using the other composers. If it had 
created a new child container it would have to explicitly register() the 
API of the child components in order to "push" them upwards. This 
explicitness ensures that it is impossible to accidentally access the 
internal structure of a component.

That's about it I think. Some implementation details in 
ComponentContainerComposer, and there's no class loading going on above 
(would be easy to add it though).

Overall, this scheme should make it possible to have lots of components, 
large components, composed components, expose what you want, hide what 
you want, and yet be reasonably easy to understand and use.

Comments, thoughts, suggestions?

/Rickard

Re: [picocontainer-dev] Re: ClassLoader Architecture

[Jeremy Boynes <jb...@apache.org>]

Rickard Öberg wrote:
> Paul Hammant wrote:
> 
>> Comp A and B (in separate classloaders) can see Common-API, which can 
>> see the classloader that contains PicoContainer who's parent is the 
>> classes in rt.jar
> 
> <snip>
> 
> One thing that I have whined about before, and I'm not sure whether it's 
> been fixed yet, is that a component A needs to be able to have an 
> internal structure (component!=1 object) and it needs to be able to 
> expose API implementations to other components without having to expose 
> its own internal structure. In the discussions so far I have not seen 
> mention of this problem (but maybe I have simply missed it).
> 

This is the role of a Configuration in Geronimo - to package together a 
pre-wired set of closely coupled GBeans and define the external 
interfaces (APIs/services offered, dependencies required). It is the 
Configurations themselves that are designed to be hot-swapped and not 
their constituents.

The implementation we have at this time is horribly naive contributing 
to the monolithic mess that is the J2EE Server configuration. Hopefully 
we can solve this sooner rather than later and avoid some of the less 
maintainable workarounds being considered.

> I have my own solution for fixing this, but I am wondering if a more 
> standard solution to this is available? (now or later) My own solution 
> is more of a best-practice use of Pico rather than an extension, and if 
> anyone is interested I can describe it in more detail.
> 

I would be very interested if you have the time.

> It really is a big big big issue once you get either loads of components 
> or large components. We have both cases in our system right now.
> 

Hear, hear - this is a big problem in Geronimo.

--
Jeremy