You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tuscany.apache.org by Ken Tam <ke...@gmail.com> on 2006/10/05 13:07:24 UTC
Packaging, usage of, and classloading for extensions & their dependencies
While working with Andy on fixing some of the Spring samples, and
taking a look at the way the war plugin has replaced the web distro, I
ran into a number of issues relating to extensions and their
dependencies.
1) It's not obvious how someone building a webapp sample expresses
their requirement for a particular extension. Given the maven focus
of our current infrastructure, the current way it seems this is done
is to just include the extension artifact as a <dependency> entry.
Does anyone have a better proposal? I think some metadata might help
here -- see the next point.
2) The war plugin doesn't understand extensions. Right now, anything
that's a <dependency> by default gets put into WEB-INF/lib, and it
might get moved into WEB-INF/tuscany/boot if the plugin determines it
only be a transitive dep of the webapp-host. This is totally
disconnected from the webapp extension loading mechanism, which
requires something to be present in WEB-INF/tuscany/extensions. So it
seems the war plugin could benefit from understanding what
dependencies are actually extensions, so it could populate it in the
right place. I would expect the standalone case to be similar (ie,
the sample build would also need to figure out to put it in the
extension directory where the DirScanExtender looks). I'm thinking
about adding metadata to the POM for extensions so that they can be
identified as such, but haven't thought about how to do this yet..
3) Extensions are loaded into a classloader that isolates them from
the application logic (the exact code in deployExtension() has FIXMEs
suggesting that while the exact classloader to be used is in flux, the
principal of isolation is true). Extensions often have their own
dependencies, which I believe fall into 2 categories:
a) deps which need to be surfaced to application logic, and thus
shouldn't be in the same isolating classloader as the extension
itself.
b) deps that ought to be hidden from application logic, and can be in
the same isolating classloader.
Taking the Spring extension as an example, the Spring framework jars
themselves would be type A deps. Right now, there is no clear way to
manage this -- for example, via maven the dep between the Spring
extension and the Spring framework jars is "just another dependency"
and so get treated just like any other jars.
I'm starting to think that it may actually be counter-productive in
the short term to attempt automation of all this -- dependency
management in general tends to be a hard problem to automate anyways
(look how long it's taken maven to attack, and it's still nowhere near
perfect). The war plugin attempts to automagically figure out where
all jars need to go, but as I think I've shown, there's a lot of
missing metadata needed to do that right. What do folks think about
simplifying it by being more explicit, and then slowly moving back to
a more automatic model?
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by ant elder <an...@gmail.com>.
On 10/5/06, Jeremy Boynes <jb...@apache.org> wrote:
<snip>
Classifying this as "signing up for an increasingly crappy
> user experience" is deriding all the work people have put into this
> already and is certainly disrespectful of the motivations this
> community. Quit whining, and thanks for volunteering to help fix
> these issues.
Its responses like this that create bad vibes and actively discourage people
from participating. I thought it was helpful Ken started this thread as I've
also just been trying to understand all this. The whole extension and
classloading thing has always been hard to workout what the plan was, i've
asked several times and never had complete answers, and I know some others
also don't understand. Even from this thread its hard to work what the
overall architecture is, and what is and isn't working - Meeraj indicates
there's still a bit to be implemented. I would help if I understood this
more, I'm sure others would as well. So we can join in would you write an
email describing the intended classloader architecture, some examples of how
to use it in the common scenarios, and the status of what does/doesn't work
right now?
...ant
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Jeremy Boynes <jb...@apache.org>.
On Oct 5, 2006, at 8:19 AM, Ken Tam wrote:
> Failure to communicate -- I was _agreeing_ with Rick, and saying that
> if we _didn't_ solve 765 (in particular supporting the pre-scan idea),
> it would result in an "increasingly crappy user experience". I _like_
> just being able to drop extensions into a dir, and don't think we
> should back off from it.
>
> I've been offline for awhile and in getting synced back up am trying
> to do things the Right Way and surface my questions on the discussion
> list -- it pains me to see my attempts at publically figuring out
> what's happened as being branded as derisive and whining.
I appreciate you bringing this up and looking for solutions. I didn't
appreciate the comments about signing up for crap.
Re-reading that message I agree that we have a failure to communicate
and that how I read it the first time is probably quite different
from what you intended. For the pains it caused you, I apologize.
--
Jeremy
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Ken Tam <ke...@gmail.com>.
On 10/5/06, Jeremy Boynes <jb...@apache.org> wrote:
> On Oct 5, 2006, at 7:10 AM, Ken Tam wrote:
> > Yeah, I haven't run into the ordering issue because I'm just dealing
> > with individual extensions, but that completely makes sense.
> >
> > It feels like if we're going to stay on the track of this maven(-like)
> > depot system for managing extensions, I would support going whole hog
> > and relying on the POM to express all dependencies -- then your
> > comment in 765:
> >
> > "If we plan on supporting just dropping in of extension jars to a
> > directory will need a means to either prescan to resolve their
> > dependencies or load all and then begin to autowire."
> >
> > would be addressed by doing the same kind of "prescan/resolution" that
> > maven does today when there are transitive and redundant dependencies.
> > Otherwise it feels like we are signing up for an increasingly crappy
> > user experience as they are forced to understand what extensions
> > depend on what other ones and in what order.
>
> We need to be clear about the difference between classloader (and
> other resource) dependencies and functional dependencies between
> components. We might use the same words but they are very different
> things.
>
> Maven's <dependency> mechanism reflects resource dependencies between
> jar files. It was originally designed to construct the classpath for
> compilation and test and has been used by assembly-style plugins to
> reflect that classpath structure in their output artifacts (like the
> way the maven war plugin uses it to populate the /WEB-INF/lib
> classpath).
>
> What Rick is running into is a functional dependency between
> components - instances that need to be registered in the right order
> due to the wires (explicit or autowire) that connect them. That's a
> very different problem. It's also a problem that we have recognized
> and discussed before and provided some workarounds for (e.g. the init-
> order configuration) although IMO we have not yet implemented a full
> solution. Classifying this as "signing up for an increasingly crappy
> user experience" is deriding all the work people have put into this
> already and is certainly disrespectful of the motivations this
> community. Quit whining, and thanks for volunteering to help fix
> these issues.
Failure to communicate -- I was _agreeing_ with Rick, and saying that
if we _didn't_ solve 765 (in particular supporting the pre-scan idea),
it would result in an "increasingly crappy user experience". I _like_
just being able to drop extensions into a dir, and don't think we
should back off from it.
I've been offline for awhile and in getting synced back up am trying
to do things the Right Way and surface my questions on the discussion
list -- it pains me to see my attempts at publically figuring out
what's happened as being branded as derisive and whining.
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Jeremy Boynes <jb...@apache.org>.
On Oct 5, 2006, at 7:10 AM, Ken Tam wrote:
> Yeah, I haven't run into the ordering issue because I'm just dealing
> with individual extensions, but that completely makes sense.
>
> It feels like if we're going to stay on the track of this maven(-like)
> depot system for managing extensions, I would support going whole hog
> and relying on the POM to express all dependencies -- then your
> comment in 765:
>
> "If we plan on supporting just dropping in of extension jars to a
> directory will need a means to either prescan to resolve their
> dependencies or load all and then begin to autowire."
>
> would be addressed by doing the same kind of "prescan/resolution" that
> maven does today when there are transitive and redundant dependencies.
> Otherwise it feels like we are signing up for an increasingly crappy
> user experience as they are forced to understand what extensions
> depend on what other ones and in what order.
We need to be clear about the difference between classloader (and
other resource) dependencies and functional dependencies between
components. We might use the same words but they are very different
things.
Maven's <dependency> mechanism reflects resource dependencies between
jar files. It was originally designed to construct the classpath for
compilation and test and has been used by assembly-style plugins to
reflect that classpath structure in their output artifacts (like the
way the maven war plugin uses it to populate the /WEB-INF/lib
classpath).
What Rick is running into is a functional dependency between
components - instances that need to be registered in the right order
due to the wires (explicit or autowire) that connect them. That's a
very different problem. It's also a problem that we have recognized
and discussed before and provided some workarounds for (e.g. the init-
order configuration) although IMO we have not yet implemented a full
solution. Classifying this as "signing up for an increasingly crappy
user experience" is deriding all the work people have put into this
already and is certainly disrespectful of the motivations this
community. Quit whining, and thanks for volunteering to help fix
these issues.
One solution proposed but not yet implemented is to load all the
extensions before starting them rather than the current way we load/
start each one in turn - that should be a fairly simple change to the
AbstractExtensionDeployer and the two subclasses that use it. A patch
for that would be welcome.
--
Jeremy
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Ken Tam <ke...@gmail.com>.
On 10/5/06, cr22rc <cr...@gmail.com> wrote:
>
>
> Ken Tam wrote:
> > While working with Andy on fixing some of the Spring samples, and
> > taking a look at the way the war plugin has replaced the web distro, I
> > ran into a number of issues relating to extensions and their
> > dependencies.
> >
> > 1) It's not obvious how someone building a webapp sample expresses
> > their requirement for a particular extension. Given the maven focus
> > of our current infrastructure, the current way it seems this is done
> > is to just include the extension artifact as a <dependency> entry.
> > Does anyone have a better proposal? I think some metadata might help
> > here -- see the next point.
> >
> > 2) The war plugin doesn't understand extensions. Right now, anything
> > that's a <dependency> by default gets put into WEB-INF/lib, and it
> > might get moved into WEB-INF/tuscany/boot if the plugin determines it
> > only be a transitive dep of the webapp-host. This is totally
> > disconnected from the webapp extension loading mechanism, which
> > requires something to be present in WEB-INF/tuscany/extensions. So it
> > seems the war plugin could benefit from understanding what
> > dependencies are actually extensions, so it could populate it in the
> > right place. I would expect the standalone case to be similar (ie,
> > the sample build would also need to figure out to put it in the
> > extension directory where the DirScanExtender looks). I'm thinking
> > about adding metadata to the POM for extensions so that they can be
> > identified as such, but haven't thought about how to do this yet..
> Maybe I'm misunderstanding but take a look at
> http://svn.apache.org/repos/asf/incubator/tuscany/java/sampleapps/bigbank/account/pom.xml
> If you look under tuscany-war-plugin in the build there are depenency elements.
> This gets me in the wars:
> WEB-INF\tuscany\extensions
> axis2-1.0-incubator-M2-SNAPSHOT.jar
> databinding-axiom-1.0-incubator-M2-SNAPSHOT.jar
> databinding-sdo-1.0-incubator-M2-SNAPSHOT.jar
> wsdl-1.0-incubator-M2-SNAPSHOT.jar
>
>
> But I admit don't know where this is documented. If it's even what your asking
> about.
That's exactly it, thanks for the pointer -- I should have grovelled
the mojo more closely.
This comment:
/* Adds the extension artifacts specified using
configuration/extensions to WEB-INF/tuscany/extensions */
actually explains it, it just wasn't clear to me that
configuration/extensions mapped to corresponding elements in the POM.
I see there's also handling for transitive dependencies of extensions,
which I suspect will rapidly run users into the classloader issues I
mentioned previously.
> > 3) Extensions are loaded into a classloader that isolates them from
> > the application logic (the exact code in deployExtension() has FIXMEs
> > suggesting that while the exact classloader to be used is in flux, the
> > principal of isolation is true). Extensions often have their own
> > dependencies, which I believe fall into 2 categories:
> >
> > a) deps which need to be surfaced to application logic, and thus
> > shouldn't be in the same isolating classloader as the extension
> > itself.
> > b) deps that ought to be hidden from application logic, and can be in
> > the same isolating classloader.
> >
> > Taking the Spring extension as an example, the Spring framework jars
> > themselves would be type A deps. Right now, there is no clear way to
> > manage this -- for example, via maven the dep between the Spring
> > extension and the Spring framework jars is "just another dependency"
> > and so get treated just like any other jars.
> >
> > I'm starting to think that it may actually be counter-productive in
> > the short term to attempt automation of all this -- dependency
> > management in general tends to be a hard problem to automate anyways
> > (look how long it's taken maven to attack, and it's still nowhere near
> > perfect). The war plugin attempts to automagically figure out where
> > all jars need to go, but as I think I've shown, there's a lot of
> > missing metadata needed to do that right. What do folks think about
> > simplifying it by being more explicit, and then slowly moving back to
> > a more automatic model?
> >
>
>
> However even though I think the war plugin can put things in the right place for
> me it all still doesn't work at runtime. Last I tried it failed cause of this
> http://issues.apache.org/jira/browse/TUSCANY-765
>
> Basically to make further progress for Big Bank I have an extremely ugly bat
> that essentially puts all back in web-inf/lib. Then in
> WEB-INF\tuscany\extensions all I have is a single scdl file that includes in a
> subdirectory the scdl of the extensions. This is so I can control the order its
> loaded. I essentially have BB now working with the fixes I put in for HTTP
> session support, and may others. So now this deploy and 765 is AFAIK the last
> hurdle to making it work "out the box."
Yeah, I haven't run into the ordering issue because I'm just dealing
with individual extensions, but that completely makes sense.
It feels like if we're going to stay on the track of this maven(-like)
depot system for managing extensions, I would support going whole hog
and relying on the POM to express all dependencies -- then your
comment in 765:
"If we plan on supporting just dropping in of extension jars to a
directory will need a means to either prescan to resolve their
dependencies or load all and then begin to autowire."
would be addressed by doing the same kind of "prescan/resolution" that
maven does today when there are transitive and redundant dependencies.
Otherwise it feels like we are signing up for an increasingly crappy
user experience as they are forced to understand what extensions
depend on what other ones and in what order.
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Jeremy Boynes <jb...@apache.org>.
As Rick said, the the war plugin has a <extensions> configuration
element do the packaging as Meeraj described here:
http://mail-archives.apache.org/mod_mbox/ws-tuscany-dev/200609.mbox/%
3c53EC6802E2BF4643BBD5BC34D2C3A38301028B1A@SPR3V4.office.local%3e
In general, you would not want to list extensions as normal pom
dependencies as that would have side effects such as adding them to
the compilation classpath or to the output war in places you did not
want.
--
Jeremy
On Oct 5, 2006, at 6:20 AM, cr22rc wrote:
>
>
> Ken Tam wrote:
>> While working with Andy on fixing some of the Spring samples, and
>> taking a look at the way the war plugin has replaced the web
>> distro, I
>> ran into a number of issues relating to extensions and their
>> dependencies.
>> 1) It's not obvious how someone building a webapp sample expresses
>> their requirement for a particular extension. Given the maven focus
>> of our current infrastructure, the current way it seems this is done
>> is to just include the extension artifact as a <dependency> entry.
>> Does anyone have a better proposal? I think some metadata might help
>> here -- see the next point.
>> 2) The war plugin doesn't understand extensions. Right now, anything
>> that's a <dependency> by default gets put into WEB-INF/lib, and it
>> might get moved into WEB-INF/tuscany/boot if the plugin determines it
>> only be a transitive dep of the webapp-host. This is totally
>> disconnected from the webapp extension loading mechanism, which
>> requires something to be present in WEB-INF/tuscany/extensions.
>> So it
>> seems the war plugin could benefit from understanding what
>> dependencies are actually extensions, so it could populate it in the
>> right place. I would expect the standalone case to be similar (ie,
>> the sample build would also need to figure out to put it in the
>> extension directory where the DirScanExtender looks). I'm thinking
>> about adding metadata to the POM for extensions so that they can be
>> identified as such, but haven't thought about how to do this yet..
> Maybe I'm misunderstanding but take a look at
> http://svn.apache.org/repos/asf/incubator/tuscany/java/sampleapps/
> bigbank/account/pom.xml
> If you look under tuscany-war-plugin in the build there are
> depenency elements.
> This gets me in the wars:
> WEB-INF\tuscany\extensions
> axis2-1.0-incubator-M2-SNAPSHOT.jar
> databinding-axiom-1.0-incubator-M2-SNAPSHOT.jar
> databinding-sdo-1.0-incubator-M2-SNAPSHOT.jar
> wsdl-1.0-incubator-M2-SNAPSHOT.jar
>
>
> But I admit don't know where this is documented. If it's even what
> your asking about.
>
>> 3) Extensions are loaded into a classloader that isolates them from
>> the application logic (the exact code in deployExtension() has FIXMEs
>> suggesting that while the exact classloader to be used is in flux,
>> the
>> principal of isolation is true). Extensions often have their own
>> dependencies, which I believe fall into 2 categories:
>> a) deps which need to be surfaced to application logic, and thus
>> shouldn't be in the same isolating classloader as the extension
>> itself.
>> b) deps that ought to be hidden from application logic, and can be in
>> the same isolating classloader.
>> Taking the Spring extension as an example, the Spring framework jars
>> themselves would be type A deps. Right now, there is no clear way to
>> manage this -- for example, via maven the dep between the Spring
>> extension and the Spring framework jars is "just another dependency"
>> and so get treated just like any other jars.
>> I'm starting to think that it may actually be counter-productive in
>> the short term to attempt automation of all this -- dependency
>> management in general tends to be a hard problem to automate anyways
>> (look how long it's taken maven to attack, and it's still nowhere
>> near
>> perfect). The war plugin attempts to automagically figure out where
>> all jars need to go, but as I think I've shown, there's a lot of
>> missing metadata needed to do that right. What do folks think about
>> simplifying it by being more explicit, and then slowly moving back to
>> a more automatic model?
>
>
> However even though I think the war plugin can put things in the
> right place for me it all still doesn't work at runtime. Last I
> tried it failed cause of this
> http://issues.apache.org/jira/browse/TUSCANY-765
>
> Basically to make further progress for Big Bank I have an extremely
> ugly bat that essentially puts all back in web-inf/lib. Then in
> WEB-INF\tuscany\extensions all I have is a single scdl file that
> includes in a subdirectory the scdl of the extensions. This is so I
> can control the order its loaded. I essentially have BB now
> working with the fixes I put in for HTTP session support, and may
> others. So now this deploy and 765 is AFAIK the last hurdle to
> making it work "out the box."
>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their
dependencies
Posted by cr22rc <cr...@gmail.com>.
Ken Tam wrote:
> While working with Andy on fixing some of the Spring samples, and
> taking a look at the way the war plugin has replaced the web distro, I
> ran into a number of issues relating to extensions and their
> dependencies.
>
> 1) It's not obvious how someone building a webapp sample expresses
> their requirement for a particular extension. Given the maven focus
> of our current infrastructure, the current way it seems this is done
> is to just include the extension artifact as a <dependency> entry.
> Does anyone have a better proposal? I think some metadata might help
> here -- see the next point.
>
> 2) The war plugin doesn't understand extensions. Right now, anything
> that's a <dependency> by default gets put into WEB-INF/lib, and it
> might get moved into WEB-INF/tuscany/boot if the plugin determines it
> only be a transitive dep of the webapp-host. This is totally
> disconnected from the webapp extension loading mechanism, which
> requires something to be present in WEB-INF/tuscany/extensions. So it
> seems the war plugin could benefit from understanding what
> dependencies are actually extensions, so it could populate it in the
> right place. I would expect the standalone case to be similar (ie,
> the sample build would also need to figure out to put it in the
> extension directory where the DirScanExtender looks). I'm thinking
> about adding metadata to the POM for extensions so that they can be
> identified as such, but haven't thought about how to do this yet..
Maybe I'm misunderstanding but take a look at
http://svn.apache.org/repos/asf/incubator/tuscany/java/sampleapps/bigbank/account/pom.xml
If you look under tuscany-war-plugin in the build there are depenency elements.
This gets me in the wars:
WEB-INF\tuscany\extensions
axis2-1.0-incubator-M2-SNAPSHOT.jar
databinding-axiom-1.0-incubator-M2-SNAPSHOT.jar
databinding-sdo-1.0-incubator-M2-SNAPSHOT.jar
wsdl-1.0-incubator-M2-SNAPSHOT.jar
But I admit don't know where this is documented. If it's even what your asking
about.
>
> 3) Extensions are loaded into a classloader that isolates them from
> the application logic (the exact code in deployExtension() has FIXMEs
> suggesting that while the exact classloader to be used is in flux, the
> principal of isolation is true). Extensions often have their own
> dependencies, which I believe fall into 2 categories:
>
> a) deps which need to be surfaced to application logic, and thus
> shouldn't be in the same isolating classloader as the extension
> itself.
> b) deps that ought to be hidden from application logic, and can be in
> the same isolating classloader.
>
> Taking the Spring extension as an example, the Spring framework jars
> themselves would be type A deps. Right now, there is no clear way to
> manage this -- for example, via maven the dep between the Spring
> extension and the Spring framework jars is "just another dependency"
> and so get treated just like any other jars.
>
> I'm starting to think that it may actually be counter-productive in
> the short term to attempt automation of all this -- dependency
> management in general tends to be a hard problem to automate anyways
> (look how long it's taken maven to attack, and it's still nowhere near
> perfect). The war plugin attempts to automagically figure out where
> all jars need to go, but as I think I've shown, there's a lot of
> missing metadata needed to do that right. What do folks think about
> simplifying it by being more explicit, and then slowly moving back to
> a more automatic model?
>
However even though I think the war plugin can put things in the right place for
me it all still doesn't work at runtime. Last I tried it failed cause of this
http://issues.apache.org/jira/browse/TUSCANY-765
Basically to make further progress for Big Bank I have an extremely ugly bat
that essentially puts all back in web-inf/lib. Then in
WEB-INF\tuscany\extensions all I have is a single scdl file that includes in a
subdirectory the scdl of the extensions. This is so I can control the order its
loaded. I essentially have BB now working with the fixes I put in for HTTP
session support, and may others. So now this deploy and 765 is AFAIK the last
hurdle to making it work "out the box."
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Jeremy Boynes <jb...@apache.org>.
On Oct 6, 2006, at 7:00 AM, Ken Tam wrote:
> Thoughts on the search order/priority for these classloader
> hierarchies? Child-before-parent has some real advantages here -- it
> lets extensions specify exactly what version of a dependency to use
> and expose to its consumers, rather than have the hosting environment
> potentially hose it by globally exposing an incompatible version.
The problem with child-before-parent is that it tends to cause too
much isolation leading to problems sharing API classes. I'd tend to
favour OSGi-type import/export for that kind of fine-grain control.
--
Jeremy
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Ken Tam <ke...@gmail.com>.
> 1) we can support ordinary jars by adding multi-parent support to the
> composite classloader. This would work in conjunction with the XML
> dependency mechanism by allowing the dependency to be loaded in an
> isolated classloader which would then be one parent of the runtime
> component and one parent of the application component. The extension
> would determine which resources it wanted to share with the
> application and which it would want to keep priviate; for example,
> the AXIOM databinding might choose to share axiom-api but would keep
> the impl private.
I like this idea as a (relatively) simple answer to how extensions
share their dependencies with application code.
I assume that the classloader associated with children components in
the app tree delegate to their parent component's classloader(s)? So
when an app component is given access by to certain resources by its
runtime implementation, that access is passed on to the app
component's child components?
Thoughts on the search order/priority for these classloader
hierarchies? Child-before-parent has some real advantages here -- it
lets extensions specify exactly what version of a dependency to use
and expose to its consumers, rather than have the hosting environment
potentially hose it by globally exposing an incompatible version.
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Jeremy Boynes <jb...@apache.org>.
On Oct 5, 2006, at 11:28 PM, Jim Marino wrote:
>> A more complex case is where a composite at some arbitrary depth
>> down the application tree wants to share a class with a system
>> component (such as an extension) that is at some other arbitrary
>> depth down the runtime tree. We have two solutions on the table
>> for this:
>>
>> 1) we can support ordinary jars by adding multi-parent support to
>> the composite classloader. This would work in conjunction with the
>> XML dependency mechanism by allowing the dependency to be loaded
>> in an isolated classloader which would then be one parent of the
>> runtime component and one parent of the application component. The
>> extension would determine which resources it wanted to share with
>> the application and which it would want to keep priviate; for
>> example, the AXIOM databinding might choose to share axiom-api but
>> would keep the impl private.
>>
> This makes sense. We will need to work out the specifics of how
> extension dependencies are marked as visible to application
> composites, probably in the dependency element. I don't think we
> need to be as granular as OSGi.
I think the goal here is for a simple solution - if people need
esoteric schemes then OSGi packaging is available.
This may be as simple as loading all dependencies into a isolated
parent anyway and having the builders add the dependencies they want
to share as parents to the application classloader. Parent-first
delegation (assuming the tree is searched before the ones added by
the builders) would result in the common classes being located. If
that works, it's easy for the end-user as they don't need to specify
anything.
>> 2) we can support OSGi bundles using the import/export semantics
>> of the OSGi classloader. This would require users and extension
>> providers to package their composites as OSGi bundles.
> I'm wondering whether we may want to take a simpler approach.
> Systems developers may not have difficulty creating OSGi bundles
> but this would be a significant complication for application
> developers (it would require them to import extension dependencies
> in a manifest). Also, this will mean that applications/extensions
> packaging would not be portable across host environments.
I think the proposal above covers both the simple case and the legacy
case (using existing jars).
With the popularity of Eclipse I see more and more artifacts will
become OSGi enabled and hence users will become more familiar with it
and tools will make it easier (if not transparent). However, that's a
future thing.
For portability, OSGi is a standard and that may make it more
acceptable to the OSOA collaboration than another packaging
mechanism. We may be able to persuade them to support both (given the
first allows the use of legacy artifacts as well). That should be a
lively debate :-)
>
> For deploying to a OSGi container, we may still want to use the
> classloader structuring above and have the Tuscany runtime deployed
> in a bundle. Applications and extensions would be deployed to
> directories accessible to the directory scanner and they would be
> loaded in the context of the runtime bundle. Tuscany would then
> take over and impose its classloading hierarchy on the composite
> trees. The runtime could use OSGi services by importing them
> through a reference configured with an OSGi binding. Likewise, the
> runtime could export services to the OSGi host through a service
> configured with the OSGi binding. Some system services (e.g. HTTP
> transport, work manager, etc.) could be OSGi services that were
> imported through the OSGi binding.
>
> The above approach would allow applications and extensions
> developers to use a common packaging and deployment model across
> runtime hosts.
I agree, particularly because it allows the portability of SCA
applications between OSGi and non-OSGI based runtimes.
The other approach is direct deployment using the OSGi host's
deployment mechanisms. I think that may be a feature provided by and
particular to the OSGi integration (perhaps even proprietary to a
specific OSGi implementation - I'd need to check how portable the
service provision/discovery has become).
--
Jeremy
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Jim Marino <jm...@myromatours.com>.
On Oct 6, 2006, at 4:16 AM, ant elder wrote:
> Where do you see all this fitting in with the M2 release? I'd like
> an M2
> release real soon, what do you think we need for that and how about
> leaving
> all the OSGi stuff for after M2?
>
Yep, I see it after M2, since I'd like to get the release out soon too.
Jim
> ...ant
>
> On 10/6/06, Jim Marino <jm...@myromatours.com> wrote:
>>
>> > For the webapp host, the default application is contained in the
>> > host environment (e.g. it contains JSPs, Servlets et al that are
>> > part of the webapp). The default composite has a classloader that
>> > is a child of the webapp one so has a classpath that is all the
>> > application code + any <dependency>'s specified in the composite
>> scdl.
>> >
>> > In this way we get the isolation we need between the application
>> > code and the runtime implementation.
>> >
>> > Where this bites is when there are resources (classes) that need to
>> > be shared between the application programming model and the
>> > implementation. The way in which that happens depends on the nature
>> > of the sharing.
>> >
>> > At one level, the easy solution is to introduce those classes into
>> > the host classloader so that they are shared between the
>> > application side and the runtime side. This is what we do with the
>> > SCA API classes, and what a user can do in the webapp host by
>> > adding the classes to the webapp classloader (e.g. by putting them
>> > in WEB-INF/classes or a jar in WEB-INF/lib). There are no problems
>> > with this if the runtime code loads classes correctly (using the
>> > appropriate classloader which is most probably not the TCCL).
>> >
>> > A more complex case is where a composite at some arbitrary depth
>> > down the application tree wants to share a class with a system
>> > component (such as an extension) that is at some other arbitrary
>> > depth down the runtime tree. We have two solutions on the table for
>> > this:
>> >
>> > 1) we can support ordinary jars by adding multi-parent support to
>> > the composite classloader. This would work in conjunction with the
>> > XML dependency mechanism by allowing the dependency to be loaded in
>> > an isolated classloader which would then be one parent of the
>> > runtime component and one parent of the application component. The
>> > extension would determine which resources it wanted to share with
>> > the application and which it would want to keep priviate; for
>> > example, the AXIOM databinding might choose to share axiom-api but
>> > would keep the impl private.
>> >
>> This makes sense. We will need to work out the specifics of how
>> extension dependencies are marked as visible to application
>> composites, probably in the dependency element. I don't think we need
>> to be as granular as OSGi.
>>
>> > 2) we can support OSGi bundles using the import/export semantics of
>> > the OSGi classloader. This would require users and extension
>> > providers to package their composites as OSGi bundles.
>> I'm wondering whether we may want to take a simpler approach. Systems
>> developers may not have difficulty creating OSGi bundles but this
>> would be a significant complication for application developers (it
>> would require them to import extension dependencies in a manifest).
>> Also, this will mean that applications/extensions packaging would not
>> be portable across host environments.
>>
>> For deploying to a OSGi container, we may still want to use the
>> classloader structuring above and have the Tuscany runtime deployed
>> in a bundle. Applications and extensions would be deployed to
>> directories accessible to the directory scanner and they would be
>> loaded in the context of the runtime bundle. Tuscany would then take
>> over and impose its classloading hierarchy on the composite trees.
>> The runtime could use OSGi services by importing them through a
>> reference configured with an OSGi binding. Likewise, the runtime
>> could export services to the OSGi host through a service configured
>> with the OSGi binding. Some system services (e.g. HTTP transport,
>> work manager, etc.) could be OSGi services that were imported through
>> the OSGi binding.
>>
>> The above approach would allow applications and extensions developers
>> to use a common packaging and deployment model across runtime hosts.
>>
>> Thoughts?
>>
>> Jim
>>
>> >
>> > IMO we need to do both - although OSGi is very flexible it brings
>> > quite a bit of baggage with it which users may not want to opt in
>> > to. Although more and more resources are making things available as
>> > OSGi bundles not everything does. We also need to support legacy
>> > applications/libraries that do not have the metadata that OSGi
>> > requires.
>> >
>> > --
>> > Jeremy
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Jeremy Boynes <jb...@apache.org>.
The release plan had osgi in the "include if its ready" category. IMO
I don't think it's ready yet so would put it in the "contrib"
category (in the source, not in the binary, YMMV).
The need for multi-parent is really driven by SDO and Spring
support. In the webapp environment the user is probably going to
need them from unmanaged code as well so they would be in the
webapp's lib directory anyway so we should not have any issues.
However, we may need to add this for the standalone environment.
--
Jeremy
On Oct 6, 2006, at 4:16 AM, ant elder wrote:
> Where do you see all this fitting in with the M2 release? I'd like
> an M2
> release real soon, what do you think we need for that and how about
> leaving
> all the OSGi stuff for after M2?
>
> ...ant
>
> On 10/6/06, Jim Marino <jm...@myromatours.com> wrote:
>>
>> > For the webapp host, the default application is contained in the
>> > host environment (e.g. it contains JSPs, Servlets et al that are
>> > part of the webapp). The default composite has a classloader that
>> > is a child of the webapp one so has a classpath that is all the
>> > application code + any <dependency>'s specified in the composite
>> scdl.
>> >
>> > In this way we get the isolation we need between the application
>> > code and the runtime implementation.
>> >
>> > Where this bites is when there are resources (classes) that need to
>> > be shared between the application programming model and the
>> > implementation. The way in which that happens depends on the nature
>> > of the sharing.
>> >
>> > At one level, the easy solution is to introduce those classes into
>> > the host classloader so that they are shared between the
>> > application side and the runtime side. This is what we do with the
>> > SCA API classes, and what a user can do in the webapp host by
>> > adding the classes to the webapp classloader (e.g. by putting them
>> > in WEB-INF/classes or a jar in WEB-INF/lib). There are no problems
>> > with this if the runtime code loads classes correctly (using the
>> > appropriate classloader which is most probably not the TCCL).
>> >
>> > A more complex case is where a composite at some arbitrary depth
>> > down the application tree wants to share a class with a system
>> > component (such as an extension) that is at some other arbitrary
>> > depth down the runtime tree. We have two solutions on the table for
>> > this:
>> >
>> > 1) we can support ordinary jars by adding multi-parent support to
>> > the composite classloader. This would work in conjunction with the
>> > XML dependency mechanism by allowing the dependency to be loaded in
>> > an isolated classloader which would then be one parent of the
>> > runtime component and one parent of the application component. The
>> > extension would determine which resources it wanted to share with
>> > the application and which it would want to keep priviate; for
>> > example, the AXIOM databinding might choose to share axiom-api but
>> > would keep the impl private.
>> >
>> This makes sense. We will need to work out the specifics of how
>> extension dependencies are marked as visible to application
>> composites, probably in the dependency element. I don't think we need
>> to be as granular as OSGi.
>>
>> > 2) we can support OSGi bundles using the import/export semantics of
>> > the OSGi classloader. This would require users and extension
>> > providers to package their composites as OSGi bundles.
>> I'm wondering whether we may want to take a simpler approach. Systems
>> developers may not have difficulty creating OSGi bundles but this
>> would be a significant complication for application developers (it
>> would require them to import extension dependencies in a manifest).
>> Also, this will mean that applications/extensions packaging would not
>> be portable across host environments.
>>
>> For deploying to a OSGi container, we may still want to use the
>> classloader structuring above and have the Tuscany runtime deployed
>> in a bundle. Applications and extensions would be deployed to
>> directories accessible to the directory scanner and they would be
>> loaded in the context of the runtime bundle. Tuscany would then take
>> over and impose its classloading hierarchy on the composite trees.
>> The runtime could use OSGi services by importing them through a
>> reference configured with an OSGi binding. Likewise, the runtime
>> could export services to the OSGi host through a service configured
>> with the OSGi binding. Some system services (e.g. HTTP transport,
>> work manager, etc.) could be OSGi services that were imported through
>> the OSGi binding.
>>
>> The above approach would allow applications and extensions developers
>> to use a common packaging and deployment model across runtime hosts.
>>
>> Thoughts?
>>
>> Jim
>>
>> >
>> > IMO we need to do both - although OSGi is very flexible it brings
>> > quite a bit of baggage with it which users may not want to opt in
>> > to. Although more and more resources are making things available as
>> > OSGi bundles not everything does. We also need to support legacy
>> > applications/libraries that do not have the metadata that OSGi
>> > requires.
>> >
>> > --
>> > Jeremy
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by ant elder <an...@gmail.com>.
Where do you see all this fitting in with the M2 release? I'd like an M2
release real soon, what do you think we need for that and how about leaving
all the OSGi stuff for after M2?
...ant
On 10/6/06, Jim Marino <jm...@myromatours.com> wrote:
>
> > For the webapp host, the default application is contained in the
> > host environment (e.g. it contains JSPs, Servlets et al that are
> > part of the webapp). The default composite has a classloader that
> > is a child of the webapp one so has a classpath that is all the
> > application code + any <dependency>'s specified in the composite scdl.
> >
> > In this way we get the isolation we need between the application
> > code and the runtime implementation.
> >
> > Where this bites is when there are resources (classes) that need to
> > be shared between the application programming model and the
> > implementation. The way in which that happens depends on the nature
> > of the sharing.
> >
> > At one level, the easy solution is to introduce those classes into
> > the host classloader so that they are shared between the
> > application side and the runtime side. This is what we do with the
> > SCA API classes, and what a user can do in the webapp host by
> > adding the classes to the webapp classloader (e.g. by putting them
> > in WEB-INF/classes or a jar in WEB-INF/lib). There are no problems
> > with this if the runtime code loads classes correctly (using the
> > appropriate classloader which is most probably not the TCCL).
> >
> > A more complex case is where a composite at some arbitrary depth
> > down the application tree wants to share a class with a system
> > component (such as an extension) that is at some other arbitrary
> > depth down the runtime tree. We have two solutions on the table for
> > this:
> >
> > 1) we can support ordinary jars by adding multi-parent support to
> > the composite classloader. This would work in conjunction with the
> > XML dependency mechanism by allowing the dependency to be loaded in
> > an isolated classloader which would then be one parent of the
> > runtime component and one parent of the application component. The
> > extension would determine which resources it wanted to share with
> > the application and which it would want to keep priviate; for
> > example, the AXIOM databinding might choose to share axiom-api but
> > would keep the impl private.
> >
> This makes sense. We will need to work out the specifics of how
> extension dependencies are marked as visible to application
> composites, probably in the dependency element. I don't think we need
> to be as granular as OSGi.
>
> > 2) we can support OSGi bundles using the import/export semantics of
> > the OSGi classloader. This would require users and extension
> > providers to package their composites as OSGi bundles.
> I'm wondering whether we may want to take a simpler approach. Systems
> developers may not have difficulty creating OSGi bundles but this
> would be a significant complication for application developers (it
> would require them to import extension dependencies in a manifest).
> Also, this will mean that applications/extensions packaging would not
> be portable across host environments.
>
> For deploying to a OSGi container, we may still want to use the
> classloader structuring above and have the Tuscany runtime deployed
> in a bundle. Applications and extensions would be deployed to
> directories accessible to the directory scanner and they would be
> loaded in the context of the runtime bundle. Tuscany would then take
> over and impose its classloading hierarchy on the composite trees.
> The runtime could use OSGi services by importing them through a
> reference configured with an OSGi binding. Likewise, the runtime
> could export services to the OSGi host through a service configured
> with the OSGi binding. Some system services (e.g. HTTP transport,
> work manager, etc.) could be OSGi services that were imported through
> the OSGi binding.
>
> The above approach would allow applications and extensions developers
> to use a common packaging and deployment model across runtime hosts.
>
> Thoughts?
>
> Jim
>
> >
> > IMO we need to do both - although OSGi is very flexible it brings
> > quite a bit of baggage with it which users may not want to opt in
> > to. Although more and more resources are making things available as
> > OSGi bundles not everything does. We also need to support legacy
> > applications/libraries that do not have the metadata that OSGi
> > requires.
> >
> > --
> > Jeremy
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
>
Re: Remove use of parent ids in poms
Posted by Jeremy Boynes <jb...@apache.org>.
It's not a "feature" of maven - we can call them what we like. It may
be a "feature" of the maven-idea-plugin. I ignore them and just have
modules for the leaf projects.
I'd like to keep the overall parent (~/pom/parent/pom.xml) that
contains the project-wide configuration but if you'd like to rename
the others then please just submit a patch.
--
Jeremy
On Oct 6, 2006, at 4:27 AM, Andy Piper wrote:
> Or is this a "feature"?
>
> At 12:22 06/10/2006, Andy Piper wrote:
>> Is there any chance we could pick unique, relevant ids for the pom
>> files? The current practice of using "parent" everywhere confuses
>> IntelliJ big time.
>>
>> Thanks
>>
>> andy
>>
>> _____________________________________________________________________
>> __
>> Notice: This email message, together with any attachments, may
>> contain
>> information of BEA Systems, Inc., its subsidiaries and
>> affiliated
>> entities, that may be confidential, proprietary, copyrighted
>> and/or
>> legally privileged, and is intended solely for the use of the
>> individual
>> or entity named in this message. If you are not the intended
>> recipient,
>> and have received this message in error, please immediately return
>> this
>> by email and then delete it.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
> ______________________________________________________________________
> _
> Notice: This email message, together with any attachments, may
> contain
> information of BEA Systems, Inc., its subsidiaries and
> affiliated
> entities, that may be confidential, proprietary, copyrighted
> and/or
> legally privileged, and is intended solely for the use of the
> individual
> or entity named in this message. If you are not the intended
> recipient,
> and have received this message in error, please immediately return
> this
> by email and then delete it.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Remove use of parent ids in poms
Posted by Andy Piper <an...@bea.com>.
Or is this a "feature"?
At 12:22 06/10/2006, Andy Piper wrote:
>Is there any chance we could pick unique, relevant ids for the pom
>files? The current practice of using "parent" everywhere confuses
>IntelliJ big time.
>
>Thanks
>
>andy
>
>_______________________________________________________________________
>Notice: This email message, together with any attachments, may contain
>information of BEA Systems, Inc., its subsidiaries and affiliated
>entities, that may be confidential, proprietary, copyrighted and/or
>legally privileged, and is intended solely for the use of the individual
>or entity named in this message. If you are not the intended recipient,
>and have received this message in error, please immediately return this
>by email and then delete it.
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>For additional commands, e-mail: tuscany-dev-help@ws.apache.org
_______________________________________________________________________
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries and affiliated
entities, that may be confidential, proprietary, copyrighted and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Remove use of parent ids in poms
Posted by Andy Piper <an...@bea.com>.
Is there any chance we could pick unique, relevant ids for the pom
files? The current practice of using "parent" everywhere confuses
IntelliJ big time.
Thanks
andy
_______________________________________________________________________
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries and affiliated
entities, that may be confidential, proprietary, copyrighted and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Jim Marino <jm...@myromatours.com>.
> For the webapp host, the default application is contained in the
> host environment (e.g. it contains JSPs, Servlets et al that are
> part of the webapp). The default composite has a classloader that
> is a child of the webapp one so has a classpath that is all the
> application code + any <dependency>'s specified in the composite scdl.
>
> In this way we get the isolation we need between the application
> code and the runtime implementation.
>
> Where this bites is when there are resources (classes) that need to
> be shared between the application programming model and the
> implementation. The way in which that happens depends on the nature
> of the sharing.
>
> At one level, the easy solution is to introduce those classes into
> the host classloader so that they are shared between the
> application side and the runtime side. This is what we do with the
> SCA API classes, and what a user can do in the webapp host by
> adding the classes to the webapp classloader (e.g. by putting them
> in WEB-INF/classes or a jar in WEB-INF/lib). There are no problems
> with this if the runtime code loads classes correctly (using the
> appropriate classloader which is most probably not the TCCL).
>
> A more complex case is where a composite at some arbitrary depth
> down the application tree wants to share a class with a system
> component (such as an extension) that is at some other arbitrary
> depth down the runtime tree. We have two solutions on the table for
> this:
>
> 1) we can support ordinary jars by adding multi-parent support to
> the composite classloader. This would work in conjunction with the
> XML dependency mechanism by allowing the dependency to be loaded in
> an isolated classloader which would then be one parent of the
> runtime component and one parent of the application component. The
> extension would determine which resources it wanted to share with
> the application and which it would want to keep priviate; for
> example, the AXIOM databinding might choose to share axiom-api but
> would keep the impl private.
>
This makes sense. We will need to work out the specifics of how
extension dependencies are marked as visible to application
composites, probably in the dependency element. I don't think we need
to be as granular as OSGi.
> 2) we can support OSGi bundles using the import/export semantics of
> the OSGi classloader. This would require users and extension
> providers to package their composites as OSGi bundles.
I'm wondering whether we may want to take a simpler approach. Systems
developers may not have difficulty creating OSGi bundles but this
would be a significant complication for application developers (it
would require them to import extension dependencies in a manifest).
Also, this will mean that applications/extensions packaging would not
be portable across host environments.
For deploying to a OSGi container, we may still want to use the
classloader structuring above and have the Tuscany runtime deployed
in a bundle. Applications and extensions would be deployed to
directories accessible to the directory scanner and they would be
loaded in the context of the runtime bundle. Tuscany would then take
over and impose its classloading hierarchy on the composite trees.
The runtime could use OSGi services by importing them through a
reference configured with an OSGi binding. Likewise, the runtime
could export services to the OSGi host through a service configured
with the OSGi binding. Some system services (e.g. HTTP transport,
work manager, etc.) could be OSGi services that were imported through
the OSGi binding.
The above approach would allow applications and extensions developers
to use a common packaging and deployment model across runtime hosts.
Thoughts?
Jim
>
> IMO we need to do both - although OSGi is very flexible it brings
> quite a bit of baggage with it which users may not want to opt in
> to. Although more and more resources are making things available as
> OSGi bundles not everything does. We also need to support legacy
> applications/libraries that do not have the metadata that OSGi
> requires.
>
> --
> Jeremy
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Jeremy Boynes <jb...@apache.org>.
It's actually even more complex than the situation that you outline
as we need to deal with a hierarchy of classloaders on both the
application and runtime sides. In general, we also will not have the
ability to control what gets added to the host environment's
classloaders that are the parent to all of these.
The way we have it structured right now we have two fairly simple
trees with a common root.
The root is the "host" classloader supplied by the host environment.
In the standalone case, it's the system classloader created to run
launcher.jar. The classpath for this is the JVM classes +
launcher.jar + the contents of its manifest classpath. This one
contains classes that we expect to be shared between the application
and the runtime such as the SCA API and the Tuscany proprietary API
(but not implementations).
In the webapp case, the host classloader is the web application one
supplied by the web container. This is explicitly the thread context
classloader in place when the runtime is initialized, which by J2EE
rules is the one used to load the webapp (including the contents of
WEB-INF/classes and WEB-INF/lib/*).
To boot the Tuscany runtime we need the classes that implement it.
This is done by creating a classloader that is a child of the host
one and which contains the "boot" classes loaded from the "boot"
directory. These are the embedded core classes such as -spi and -core.
We boot the system composite ("tuscany.system") as a composite under
that classloader. As a composite it has its own classloader whose
parent classloader is the classloader of the parent composite. This
allows us to support composites that are packaged in a way that they
contribute Java resources e.g. a composite packaged as a jar file. We
also allow a composite to extend the classpath of that classloader
through <dependency> elements in its scdl.
The extension deployers are components that are part of the
"tuscany.system" composite that add additional children to it (the
extensions). Each child is a component implemented by a composite and
as such gets another child classloader.
On the application side, we deploy the "default" application
component as a composite on the application side of the component
tree. As a composite it also has its own classloader, one whose
classpath is that of the application.
For the webapp host, the default application is contained in the host
environment (e.g. it contains JSPs, Servlets et al that are part of
the webapp). The default composite has a classloader that is a child
of the webapp one so has a classpath that is all the application code
+ any <dependency>'s specified in the composite scdl.
In this way we get the isolation we need between the application code
and the runtime implementation.
Where this bites is when there are resources (classes) that need to
be shared between the application programming model and the
implementation. The way in which that happens depends on the nature
of the sharing.
At one level, the easy solution is to introduce those classes into
the host classloader so that they are shared between the application
side and the runtime side. This is what we do with the SCA API
classes, and what a user can do in the webapp host by adding the
classes to the webapp classloader (e.g. by putting them in WEB-INF/
classes or a jar in WEB-INF/lib). There are no problems with this if
the runtime code loads classes correctly (using the appropriate
classloader which is most probably not the TCCL).
A more complex case is where a composite at some arbitrary depth down
the application tree wants to share a class with a system component
(such as an extension) that is at some other arbitrary depth down the
runtime tree. We have two solutions on the table for this:
1) we can support ordinary jars by adding multi-parent support to the
composite classloader. This would work in conjunction with the XML
dependency mechanism by allowing the dependency to be loaded in an
isolated classloader which would then be one parent of the runtime
component and one parent of the application component. The extension
would determine which resources it wanted to share with the
application and which it would want to keep priviate; for example,
the AXIOM databinding might choose to share axiom-api but would keep
the impl private.
2) we can support OSGi bundles using the import/export semantics of
the OSGi classloader. This would require users and extension
providers to package their composites as OSGi bundles.
IMO we need to do both - although OSGi is very flexible it brings
quite a bit of baggage with it which users may not want to opt in to.
Although more and more resources are making things available as OSGi
bundles not everything does. We also need to support legacy
applications/libraries that do not have the metadata that OSGi requires.
--
Jeremy
On Oct 5, 2006, at 7:34 AM, Ken Tam wrote:
> On 10/5/06, ant elder <an...@gmail.com> wrote:
>> I agree its not clear how all this is supposed to work, Venkat and
>> I have
>> also both just being struggling with it.
>>
>> To be complete, adding to your (a) and (b) in (3) there's also a
>> (c) where
>> both the extension and application logic may share a dependency.
>> Examples of
>> this are js.jar for the JavaScript extension, or AXIOM if an
>> application is
>> using OMElements and the Axis2 WS binding.
>
> I think your (c) is the same as my (a) ?
>
>> > a) deps which need to be surfaced to application logic, and thus
>> > shouldn't be in the same isolating classloader as the extension
>> > itself.
>
> Is there a statement somewhere (wiki, mail, comments, anything other
> than code) about the classloader architecture? Here's a simple
> statement based on my understanding:
>
> For any given hosting environment, there are 2 important "root"
> classloaders
> a) an app-visible one (i.e. where the OSOA API jar goes, where users
> might put their own code and libraries they use). This is typically
> defined by the host environment -- for example, in a servlet container
> it would be the classloader that sees (WEB-INF/classes &&
> WEB-INF/lib).
> b) a Tuscany runtime one (i.e. where the Tuscany core jars go). Given
> that extension classloaders have this one as an ancestor, it seems
> like this one must have the app-visible one as an ancestor.
>
>
>> Could you give an example for what you mean by "simplifying it by
>> being more
>> explicit"?
>
> I think we'd need to formalize the concept of the 2 classloader
> "roots" I just described, and then expose to users the idea of
> specifying explicitly where each dependency of an extension ought to
> go. Transitively, if some dependency gets "pulled up" into the
> app-visible classloader, then that overrides any other reference to
> that dependency that places it in the Tuscany runtime one. So.. let's
> say the Spring extension uses the Spring framework jars, which in turn
> use foobar. The extension's dep on the Spring framework jars would
> have to have metadata indicating that the dep should be resolved into
> an app visible classloader. Because of that, foobar is also resolved
> into the app visible classloader, regardless of what some other
> extension that might depend on foo might say.
>
>
>
>>
>> ...ant
>>
>> On 10/5/06, Ken Tam <ke...@gmail.com> wrote:
>> >
>> > While working with Andy on fixing some of the Spring samples, and
>> > taking a look at the way the war plugin has replaced the web
>> distro, I
>> > ran into a number of issues relating to extensions and their
>> > dependencies.
>> >
>> > 1) It's not obvious how someone building a webapp sample expresses
>> > their requirement for a particular extension. Given the maven
>> focus
>> > of our current infrastructure, the current way it seems this is
>> done
>> > is to just include the extension artifact as a <dependency> entry.
>> > Does anyone have a better proposal? I think some metadata might
>> help
>> > here -- see the next point.
>> >
>> > 2) The war plugin doesn't understand extensions. Right now,
>> anything
>> > that's a <dependency> by default gets put into WEB-INF/lib, and it
>> > might get moved into WEB-INF/tuscany/boot if the plugin
>> determines it
>> > only be a transitive dep of the webapp-host. This is totally
>> > disconnected from the webapp extension loading mechanism, which
>> > requires something to be present in WEB-INF/tuscany/extensions.
>> So it
>> > seems the war plugin could benefit from understanding what
>> > dependencies are actually extensions, so it could populate it in
>> the
>> > right place. I would expect the standalone case to be similar (ie,
>> > the sample build would also need to figure out to put it in the
>> > extension directory where the DirScanExtender looks). I'm thinking
>> > about adding metadata to the POM for extensions so that they can be
>> > identified as such, but haven't thought about how to do this yet..
>> >
>> > 3) Extensions are loaded into a classloader that isolates them from
>> > the application logic (the exact code in deployExtension() has
>> FIXMEs
>> > suggesting that while the exact classloader to be used is in
>> flux, the
>> > principal of isolation is true). Extensions often have their own
>> > dependencies, which I believe fall into 2 categories:
>> >
>> > a) deps which need to be surfaced to application logic, and thus
>> > shouldn't be in the same isolating classloader as the extension
>> > itself.
>> > b) deps that ought to be hidden from application logic, and can
>> be in
>> > the same isolating classloader.
>> >
>> > Taking the Spring extension as an example, the Spring framework
>> jars
>> > themselves would be type A deps. Right now, there is no clear
>> way to
>> > manage this -- for example, via maven the dep between the Spring
>> > extension and the Spring framework jars is "just another
>> dependency"
>> > and so get treated just like any other jars.
>> >
>> > I'm starting to think that it may actually be counter-productive in
>> > the short term to attempt automation of all this -- dependency
>> > management in general tends to be a hard problem to automate
>> anyways
>> > (look how long it's taken maven to attack, and it's still
>> nowhere near
>> > perfect). The war plugin attempts to automagically figure out
>> where
>> > all jars need to go, but as I think I've shown, there's a lot of
>> > missing metadata needed to do that right. What do folks think
>> about
>> > simplifying it by being more explicit, and then slowly moving
>> back to
>> > a more automatic model?
>> >
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
>> > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>> >
>> >
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by Ken Tam <ke...@gmail.com>.
On 10/5/06, ant elder <an...@gmail.com> wrote:
> I agree its not clear how all this is supposed to work, Venkat and I have
> also both just being struggling with it.
>
> To be complete, adding to your (a) and (b) in (3) there's also a (c) where
> both the extension and application logic may share a dependency. Examples of
> this are js.jar for the JavaScript extension, or AXIOM if an application is
> using OMElements and the Axis2 WS binding.
I think your (c) is the same as my (a) ?
> > a) deps which need to be surfaced to application logic, and thus
> > shouldn't be in the same isolating classloader as the extension
> > itself.
Is there a statement somewhere (wiki, mail, comments, anything other
than code) about the classloader architecture? Here's a simple
statement based on my understanding:
For any given hosting environment, there are 2 important "root" classloaders
a) an app-visible one (i.e. where the OSOA API jar goes, where users
might put their own code and libraries they use). This is typically
defined by the host environment -- for example, in a servlet container
it would be the classloader that sees (WEB-INF/classes &&
WEB-INF/lib).
b) a Tuscany runtime one (i.e. where the Tuscany core jars go). Given
that extension classloaders have this one as an ancestor, it seems
like this one must have the app-visible one as an ancestor.
> Could you give an example for what you mean by "simplifying it by being more
> explicit"?
I think we'd need to formalize the concept of the 2 classloader
"roots" I just described, and then expose to users the idea of
specifying explicitly where each dependency of an extension ought to
go. Transitively, if some dependency gets "pulled up" into the
app-visible classloader, then that overrides any other reference to
that dependency that places it in the Tuscany runtime one. So.. let's
say the Spring extension uses the Spring framework jars, which in turn
use foobar. The extension's dep on the Spring framework jars would
have to have metadata indicating that the dep should be resolved into
an app visible classloader. Because of that, foobar is also resolved
into the app visible classloader, regardless of what some other
extension that might depend on foo might say.
>
> ...ant
>
> On 10/5/06, Ken Tam <ke...@gmail.com> wrote:
> >
> > While working with Andy on fixing some of the Spring samples, and
> > taking a look at the way the war plugin has replaced the web distro, I
> > ran into a number of issues relating to extensions and their
> > dependencies.
> >
> > 1) It's not obvious how someone building a webapp sample expresses
> > their requirement for a particular extension. Given the maven focus
> > of our current infrastructure, the current way it seems this is done
> > is to just include the extension artifact as a <dependency> entry.
> > Does anyone have a better proposal? I think some metadata might help
> > here -- see the next point.
> >
> > 2) The war plugin doesn't understand extensions. Right now, anything
> > that's a <dependency> by default gets put into WEB-INF/lib, and it
> > might get moved into WEB-INF/tuscany/boot if the plugin determines it
> > only be a transitive dep of the webapp-host. This is totally
> > disconnected from the webapp extension loading mechanism, which
> > requires something to be present in WEB-INF/tuscany/extensions. So it
> > seems the war plugin could benefit from understanding what
> > dependencies are actually extensions, so it could populate it in the
> > right place. I would expect the standalone case to be similar (ie,
> > the sample build would also need to figure out to put it in the
> > extension directory where the DirScanExtender looks). I'm thinking
> > about adding metadata to the POM for extensions so that they can be
> > identified as such, but haven't thought about how to do this yet..
> >
> > 3) Extensions are loaded into a classloader that isolates them from
> > the application logic (the exact code in deployExtension() has FIXMEs
> > suggesting that while the exact classloader to be used is in flux, the
> > principal of isolation is true). Extensions often have their own
> > dependencies, which I believe fall into 2 categories:
> >
> > a) deps which need to be surfaced to application logic, and thus
> > shouldn't be in the same isolating classloader as the extension
> > itself.
> > b) deps that ought to be hidden from application logic, and can be in
> > the same isolating classloader.
> >
> > Taking the Spring extension as an example, the Spring framework jars
> > themselves would be type A deps. Right now, there is no clear way to
> > manage this -- for example, via maven the dep between the Spring
> > extension and the Spring framework jars is "just another dependency"
> > and so get treated just like any other jars.
> >
> > I'm starting to think that it may actually be counter-productive in
> > the short term to attempt automation of all this -- dependency
> > management in general tends to be a hard problem to automate anyways
> > (look how long it's taken maven to attack, and it's still nowhere near
> > perfect). The war plugin attempts to automagically figure out where
> > all jars need to go, but as I think I've shown, there's a lot of
> > missing metadata needed to do that right. What do folks think about
> > simplifying it by being more explicit, and then slowly moving back to
> > a more automatic model?
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> > For additional commands, e-mail: tuscany-dev-help@ws.apache.org
> >
> >
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org
Re: Packaging, usage of, and classloading for extensions & their dependencies
Posted by ant elder <an...@gmail.com>.
I agree its not clear how all this is supposed to work, Venkat and I have
also both just being struggling with it.
To be complete, adding to your (a) and (b) in (3) there's also a (c) where
both the extension and application logic may share a dependency. Examples of
this are js.jar for the JavaScript extension, or AXIOM if an application is
using OMElements and the Axis2 WS binding.
Could you give an example for what you mean by "simplifying it by being more
explicit"?
...ant
On 10/5/06, Ken Tam <ke...@gmail.com> wrote:
>
> While working with Andy on fixing some of the Spring samples, and
> taking a look at the way the war plugin has replaced the web distro, I
> ran into a number of issues relating to extensions and their
> dependencies.
>
> 1) It's not obvious how someone building a webapp sample expresses
> their requirement for a particular extension. Given the maven focus
> of our current infrastructure, the current way it seems this is done
> is to just include the extension artifact as a <dependency> entry.
> Does anyone have a better proposal? I think some metadata might help
> here -- see the next point.
>
> 2) The war plugin doesn't understand extensions. Right now, anything
> that's a <dependency> by default gets put into WEB-INF/lib, and it
> might get moved into WEB-INF/tuscany/boot if the plugin determines it
> only be a transitive dep of the webapp-host. This is totally
> disconnected from the webapp extension loading mechanism, which
> requires something to be present in WEB-INF/tuscany/extensions. So it
> seems the war plugin could benefit from understanding what
> dependencies are actually extensions, so it could populate it in the
> right place. I would expect the standalone case to be similar (ie,
> the sample build would also need to figure out to put it in the
> extension directory where the DirScanExtender looks). I'm thinking
> about adding metadata to the POM for extensions so that they can be
> identified as such, but haven't thought about how to do this yet..
>
> 3) Extensions are loaded into a classloader that isolates them from
> the application logic (the exact code in deployExtension() has FIXMEs
> suggesting that while the exact classloader to be used is in flux, the
> principal of isolation is true). Extensions often have their own
> dependencies, which I believe fall into 2 categories:
>
> a) deps which need to be surfaced to application logic, and thus
> shouldn't be in the same isolating classloader as the extension
> itself.
> b) deps that ought to be hidden from application logic, and can be in
> the same isolating classloader.
>
> Taking the Spring extension as an example, the Spring framework jars
> themselves would be type A deps. Right now, there is no clear way to
> manage this -- for example, via maven the dep between the Spring
> extension and the Spring framework jars is "just another dependency"
> and so get treated just like any other jars.
>
> I'm starting to think that it may actually be counter-productive in
> the short term to attempt automation of all this -- dependency
> management in general tends to be a hard problem to automate anyways
> (look how long it's taken maven to attack, and it's still nowhere near
> perfect). The war plugin attempts to automagically figure out where
> all jars need to go, but as I think I've shown, there's a lot of
> missing metadata needed to do that right. What do folks think about
> simplifying it by being more explicit, and then slowly moving back to
> a more automatic model?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
> For additional commands, e-mail: tuscany-dev-help@ws.apache.org
>
>
Re: Packaging, usage of, and classloading for extensions &
their dependencies
Posted by Andy Piper <an...@bea.com>.
At 12:07 05/10/2006, Ken Tam wrote:
>missing metadata needed to do that right. What do folks think about
>simplifying it by being more explicit, and then slowly moving back to
>a more automatic model?
Gets my vote.
andy
_______________________________________________________________________
Notice: This email message, together with any attachments, may contain
information of BEA Systems, Inc., its subsidiaries and affiliated
entities, that may be confidential, proprietary, copyrighted and/or
legally privileged, and is intended solely for the use of the individual
or entity named in this message. If you are not the intended recipient,
and have received this message in error, please immediately return this
by email and then delete it.
---------------------------------------------------------------------
To unsubscribe, e-mail: tuscany-dev-unsubscribe@ws.apache.org
For additional commands, e-mail: tuscany-dev-help@ws.apache.org