You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@river.apache.org by Gregg Wonderly <gr...@wonderly.org> on 2009/10/14 18:42:08 UTC
Separating out RMIClassLoader dependencies
As I try to put together some netbeans modules for Jini development tasks, I am
fighting with netbeans classloader use alot (and not actually winning it seems).
So I am thinking (actually I'm trying some of this out) about whether it would
make sense to help with OSGi and other classloader integration as well, if we
could put an SPI in front of the use of RMIClassLoader.loadClass() etc so that
different environments could provide more deliberate control over where classes
are resolved.
The problem I am having in netbeans is that I need the PreferredClassProvider
and jsk-resources.jar which provides the Jini RMIClassLoaderSpi to be in the JVM
system class loader to make it active. Yet that class loader is invisible to
the netbeans module subsystem because it provides brickwall separation of the
modules from each other through separate classloaders that will delegate to the
netbeans system class loader. All of the delegation back and forth is what I am
fighting with.
What I am playing with is creating a static class that mimics the
RMIClassLoaderSpi by providing the same methods, but which then also provides a
security controlled setter to set the implementation. An interface would then
document the methods that this SPI requires, and then implementation of that
interface could be "set" on this static class.
The simple nature of this change means that we can just replace RMIClassLoader
references with the static class reference, and provide a simple implementation
of the interface that just uses RMIClassLoader.xxxx() calls for implementation.
That would put the code in a position to accept pluggable implementations of
class resolution I believe.
However, I'm still trying to think through how PreferredClassProvider and
ClassLoading interact with RMIClassLoaderSPI to see if I've forgotten something big.
I'm curious if anyone from the Jini team at Sun has any insight into whether
this would really make it possible to better control which class loaders are used.
Right now, there are several places in the Jini code where null ClassLoader
references end up redirecting loading into the netbeans context class loader
which is almost always the netbeans system class loader, and I'm having a hard
time keeping the codebase loader visible.
Anyone have thoughts or questions?
Gregg Wonderly
Re: Separating out RMIClassLoader dependencies
Posted by richard nicholson <ri...@paremus.com>.
Niclas / All (interested)
I've asked one of the Paremus engineering team to comment back on this.
We're busy at present - two imminent releases - so may be a couple of
weeks.
Best Wishes
Richard
On 15 Oct 2009, at 03:41, Niclas Hedhman wrote:
> On Thu, Oct 15, 2009 at 12:42 AM, Gregg Wonderly
> <gr...@wonderly.org> wrote:
>
>> Anyone have thoughts or questions?
>
> For OSGi (maybe Netbeans as well) there might be additional
> "problems", since "loading a class" is not a straight forward process
> of calling a static provider. In OSGi, each "module" is wired up with
> a "class space" but the same class name (difference version, different
> provider, etc) may be in another "class space" in other modules. So,
> to fulfill a contract where a static method gets called from
> "anywhere" and I need to figure out which class to load is near
> impossible (at least in a platform independent way) in OSGi, as you
> will have to figure out which bundle (module) the caller reside in, in
> an environment where multiple versions of the same class can exist.
> (The reason this is near impossible is that the OSGi spec doesn't have
> any Classloader API, from which the Bundle can be retrieved.)
>
> "Class space" is a complex and powerful concept in OSGi, which
> basically try to ensure that there are no class version clashes and
> yet provide the class visibility needed.
>
> OTOH, Paremus has shown that
>
> a) Locally available classes can be used,
> b) Remote classes be downloaded and wrapped in temporary bundles with
> OSGi contracts honored.
> c) PreferredClassLoader is not needed.
>
> So, perhaps your ambitions are too great and that you can/should
> ignore OSGi's needs.
>
>
> Cheers
> --
> Niclas Hedhman, Software Developer
> http://www.qi4j.org - New Energy for Java
>
> I live here; http://tinyurl.com/2qq9er
> I work here; http://tinyurl.com/2ymelc
> I relax here; http://tinyurl.com/2cgsug
Re: Cooperative Distributed computing - Versioning over time - Container
Metadata.
Posted by Peter Firmstone <ji...@zeus.net.au>.
Calum Shaw-Mackay wrote:
>> Interesting.
>>
>> Perhaps Marshalled object instances should also be decorated.
>>
>
> Yeah you'd need somewhere to start, and as I mentioned below about
> transparency, you'd possibly need a
> DEFAULT or CURRENT version, because if you're bringing in an object
> foreign to the VM, and you're using
> static analysis, you'd need to have the bytecodes available to
> actually compare it against, having an initial
> version identifier would give it a head start in terms of performance,
> and you don't want to brute-force
> hammer the network bringing in multiple dl jars, for a class that has
> not been seen in any version in this JVM session
>
> I think the more 'hints' you could provide to the classloader, through
> either compatible version ranges or soft links
> the better.
>
>
>>> Whichever way you want to tackle this, you'll need it to be as
>>> transparent a mechanism as possible so that service/bundles/clients
>>> that don;t specify versions work as they always have,
>>>
>>>
>> Thanks Calum, good point, when version metadata doesn't exist, the results
>> from Static Analysis alone could be utilised to find API compatible import
>> packages. I guess classes lacking version metadata could be treated as
>> version 0. If someone later decides to add version metadata, perhaps after
>> correcting a bug, the higher versioned package would be preferred, provided
>> it remains API compatible and shares the same fully qualified package name.
>>
>
> Yeah I'd say a DEFAULT (i.e. 0, although that itself may go through
> changes in time) and perhaps symlink-type versioning including a
> CURRENT.
>
I could use the class file modification dates from the jar files, so
where more than one package of the same name exists, where both have an
API that satisfy the dependency requirements, where version metadata
isn't provided, the later modified file is preferred.
> In a way, what you don't want is to break compatibility unnecessarily.
>
> Calum
>
>
Re: Cooperative Distributed computing - Versioning over time - Container
Metadata.
Posted by Peter Firmstone <ji...@zeus.net.au>.
Gregg Wonderly wrote:
> My changes to reggie to provide "never preferred classes" changed
> MarshalledWrapper and EntryRep to implement
>
> public interface MarshalledDataAccess<T> {
> public T getUnmarshalledData() throws IOException,
> InstantiationException,
> ClassNotFoundException, IllegalAccessException ;
> public List<String> getClassNames();
> }
>
> There is still some other work that I need to finalize in these
> changes to get service objects to always have this information (there
> are complex threads of execution and MarshalledObject use).
>
> I made EntryRep work, and that was the big thing that I needed, so I
> stopped at that point.
>
> Creating an interface, such as the above, and putting it into play
> would be helpful I think.
>
> I also still believe that we really should change lookup to use
> unmarshalled data objects (these changes are visible at
> http://reef.dev.java.net) so that we can then plug in unmarshalling
> contexts and metadata transport pieces to make services have enough
> plug-ability that we can provide for registration and discovery of
> other types of "enviroments" than just what DiscoveryV1 and V2 provide.
>
> Gregg Wonderly
You know what, your right. Reggie is implementation specific. We need
to change that.
We need a place on svn for River 3.0 where we can make these changes and
experiment, perhaps using a minimal subset of the current codebase for
clarity while we experiment.
What components we should initially include?
The AR2 codebase is ready for release apart from some minor changes to
testing that I haven't quite gotten around to yet, no good holding up
productivity of fertile minds in the mean time... Its safe to branch.
Cheers,
Peter.
>
> Calum Shaw-Mackay wrote:
>>> Interesting.
>>>
>>> Perhaps Marshalled object instances should also be decorated.
>>
>> Yeah you'd need somewhere to start, and as I mentioned below about
>> transparency, you'd possibly need a
>> DEFAULT or CURRENT version, because if you're bringing in an object
>> foreign to the VM, and you're using
>> static analysis, you'd need to have the bytecodes available to
>> actually compare it against, having an initial
>> version identifier would give it a head start in terms of performance,
>> and you don't want to brute-force
>> hammer the network bringing in multiple dl jars, for a class that has
>> not been seen in any version in this JVM session
>>
>> I think the more 'hints' you could provide to the classloader, through
>> either compatible version ranges or soft links
>> the better.
>>
>>>> Whichever way you want to tackle this, you'll need it to be as
>>>> transparent a mechanism as possible so that service/bundles/clients
>>>> that don;t specify versions work as they always have,
>>>>
>>> Thanks Calum, good point, when version metadata doesn't exist, the
>>> results
>>> from Static Analysis alone could be utilised to find API compatible
>>> import
>>> packages. I guess classes lacking version metadata could be treated as
>>> version 0. If someone later decides to add version metadata,
>>> perhaps after
>>> correcting a bug, the higher versioned package would be preferred,
>>> provided
>>> it remains API compatible and shares the same fully qualified
>>> package name.
>>
>> Yeah I'd say a DEFAULT (i.e. 0, although that itself may go through
>> changes in time) and perhaps symlink-type versioning including a
>> CURRENT.
>>
>> In a way, what you don't want is to break compatibility unnecessarily.
>>
>> Calum
>>
>
>
Re: Cooperative Distributed computing - Versioning over time - Container
Metadata.
Posted by Gregg Wonderly <ge...@cox.net>.
My changes to reggie to provide "never preferred classes" changed
MarshalledWrapper and EntryRep to implement
public interface MarshalledDataAccess<T> {
public T getUnmarshalledData() throws IOException, InstantiationException,
ClassNotFoundException, IllegalAccessException ;
public List<String> getClassNames();
}
There is still some other work that I need to finalize in these changes to get
service objects to always have this information (there are complex threads of
execution and MarshalledObject use).
I made EntryRep work, and that was the big thing that I needed, so I stopped at
that point.
Creating an interface, such as the above, and putting it into play would be
helpful I think.
I also still believe that we really should change lookup to use unmarshalled
data objects (these changes are visible at http://reef.dev.java.net) so that we
can then plug in unmarshalling contexts and metadata transport pieces to make
services have enough plug-ability that we can provide for registration and
discovery of other types of "enviroments" than just what DiscoveryV1 and V2 provide.
Gregg Wonderly
Calum Shaw-Mackay wrote:
>> Interesting.
>>
>> Perhaps Marshalled object instances should also be decorated.
>
> Yeah you'd need somewhere to start, and as I mentioned below about
> transparency, you'd possibly need a
> DEFAULT or CURRENT version, because if you're bringing in an object
> foreign to the VM, and you're using
> static analysis, you'd need to have the bytecodes available to
> actually compare it against, having an initial
> version identifier would give it a head start in terms of performance,
> and you don't want to brute-force
> hammer the network bringing in multiple dl jars, for a class that has
> not been seen in any version in this JVM session
>
> I think the more 'hints' you could provide to the classloader, through
> either compatible version ranges or soft links
> the better.
>
>>> Whichever way you want to tackle this, you'll need it to be as
>>> transparent a mechanism as possible so that service/bundles/clients
>>> that don;t specify versions work as they always have,
>>>
>> Thanks Calum, good point, when version metadata doesn't exist, the results
>> from Static Analysis alone could be utilised to find API compatible import
>> packages. I guess classes lacking version metadata could be treated as
>> version 0. If someone later decides to add version metadata, perhaps after
>> correcting a bug, the higher versioned package would be preferred, provided
>> it remains API compatible and shares the same fully qualified package name.
>
> Yeah I'd say a DEFAULT (i.e. 0, although that itself may go through
> changes in time) and perhaps symlink-type versioning including a
> CURRENT.
>
> In a way, what you don't want is to break compatibility unnecessarily.
>
> Calum
>
Re: Cooperative Distributed computing - Versioning over time -
Container Metadata.
Posted by Calum Shaw-Mackay <ca...@gmail.com>.
>
> Interesting.
>
> Perhaps Marshalled object instances should also be decorated.
Yeah you'd need somewhere to start, and as I mentioned below about
transparency, you'd possibly need a
DEFAULT or CURRENT version, because if you're bringing in an object
foreign to the VM, and you're using
static analysis, you'd need to have the bytecodes available to
actually compare it against, having an initial
version identifier would give it a head start in terms of performance,
and you don't want to brute-force
hammer the network bringing in multiple dl jars, for a class that has
not been seen in any version in this JVM session
I think the more 'hints' you could provide to the classloader, through
either compatible version ranges or soft links
the better.
>>
>> Whichever way you want to tackle this, you'll need it to be as
>> transparent a mechanism as possible so that service/bundles/clients
>> that don;t specify versions work as they always have,
>>
>
> Thanks Calum, good point, when version metadata doesn't exist, the results
> from Static Analysis alone could be utilised to find API compatible import
> packages. I guess classes lacking version metadata could be treated as
> version 0. If someone later decides to add version metadata, perhaps after
> correcting a bug, the higher versioned package would be preferred, provided
> it remains API compatible and shares the same fully qualified package name.
Yeah I'd say a DEFAULT (i.e. 0, although that itself may go through
changes in time) and perhaps symlink-type versioning including a
CURRENT.
In a way, what you don't want is to break compatibility unnecessarily.
Calum
Re: Cooperative Distributed computing - Versioning over time - Container
Metadata.
Posted by Peter Firmstone <ji...@zeus.net.au>.
Calum Shaw-Mackay wrote:
> This sounds very like Newton......
>
> Another way might be to provide a new URL Handler and having a Version
> ClassLoader higher up the classloader stack.
>
I'd like to make the traditional http codebase obsolete, or at a minimum
have a codebase service act as an intermediatory on its behalf.
Traditional http codebases are a source of pain. I want something
simpler. This would allow utilisation of byte code analysis to build
class mirrors that the classloading infrastructure could use to resolve
compatible packages, probably with a version classloader higher up the
stack as you suggest, which keeps track of where all currently loaded
packages, like classworlds.
> Although this doesn't really help in terms of attaching the version
> required by a caller when trying to find a particular version, what
> you might need is a decorated
> classloader, that is built on the metadata supplied by the
> service/client, with a version classloader acting as it's parent, so
> when a class is requested by the service/client, it calls the
> decorated classloader, which contains the metadata, which in turn
> requests loadClass(name, version) from the Versioned ClassLoader.
>
Interesting.
Perhaps Marshalled object instances should also be decorated.
> Whichever way you want to tackle this, you'll need it to be as
> transparent a mechanism as possible so that service/bundles/clients
> that don;t specify versions work as they always have,
>
Thanks Calum, good point, when version metadata doesn't exist, the
results from Static Analysis alone could be utilised to find API
compatible import packages. I guess classes lacking version metadata
could be treated as version 0. If someone later decides to add version
metadata, perhaps after correcting a bug, the higher versioned package
would be preferred, provided it remains API compatible and shares the
same fully qualified package name.
> --Calum
>
> 2009/10/20 Peter Firmstone <ji...@zeus.net.au>:
>
>> N.B.
>>
>> If we were able to utilise OSGi bundles, I can't then see why there couldn't
>> be a way of setting up some kind of Bridge or Facade pattern to interoperate
>> with OSGi. I'm only hypothesising, I don't have any particular
>> implementation in mind. The advantage here would be the possibility of using
>> common bundle code for both environments.
>>
>> Peter Firmstone wrote:
>>
>>> Let me put it another way:
>>>
>>> I want to work toward a cooperative distributed computing platform using
>>> Java & Jini / River, continuing from where Sun left, using lessons learned.
>>>
>>> 1. I'd like to set up a class loader framework building on
>>> classworlds, to segregate incompatible class versions with the
>>> same name, while allowing compatible implementations to
>>> interoperate, based on Class loader visibility.
>>> 2. I'd like to utilise the ASM bytecode library to perform static
>>> analysis on bytecode to identify Package & Class API to determine
>>> compatibility.
>>> 3. I'd like to solve the lost code base problem using codebase
>>> services combined with the suggested Class loading framework.
>>> 4. I'd like to use codebase servcies to provide codebase proxy's for
>>> often disconnected clients.
>>> 5. I'd like to allow bytecode to evolve organically by taking
>>> advantage of the flexibility of binary compatibility.
>>> 6. I'd like some metadata to identify at the minimum ,Package
>>> Versions, I could generate checksums based on Package API, however
>>> this wouldn't identify which version is the latest, only which
>>> Package implementations can substitute another's API where those
>>> packages share the same name. Or identify packages with the same
>>> name that are incompatible and segregate these in separate
>>> Classloaders.
>>> 7. If possible due to concerns about the performance impact of many
>>> Classloaders, group some packages together when the author doesn't
>>> intend those packages to evolve separately. Eg libraries.
>>> 8. I'd like to make it possible to transfer class version and package
>>> version information with serialized object instances for
>>> compatible re-instantiation with the latest evolution of
>>> compatible classes. All suggestions are welcome.
>>>
>>> Does anyone have any suggestions for suitable existing formats for
>>> Containers with Version Metadata that might be utilised? If so what issues
>>> are there to be aware of? What are the arguments for and Against?
>>>
>>> Note, I'm aware of the existing Java Version Spec that applies to
>>> packages.
>>>
>>> I believe I made the mistake of suggesting a solution without properly
>>> formulating or posing the problem, my apologies for any confusion this might
>>> have caused, I should have raised a new thread / topic, In the original
>>> thread, Gregg is looking at classloader dependency issues implementing a
>>> plugin for Netbeans to make River more accessible to IDE developers, please
>>> feel free to make suggestions, we need your help.
>>>
>>> I made a suggestion earlier to utilise the OSGi bundle metadata
>>> specification, these are ordinary jar files with package version, exports
>>> and dependency metadata. They are otherwise ordinary plain old jar
>>> archives, quite useable as a jar file containing java class files that can
>>> be utilised without the OSGi framework, which is what I was suggesting. I
>>> attempted to imply that the exported packages could have their API
>>> identified and shared in a compatible manner with other packages, using
>>> Class loader isolation perhaps this is not a suitable solution due to the
>>> confusion that it would cause with developers expecting an OSGi
>>> implementation, utilising OSGi service interfaces. If this format was to be
>>> utilised, it might have to be renamed, to River Codebase Bundles (for
>>> uploading to codebases) whilst sharing the same format to avoid confusion.
>>> It might be a subset of the OSGi bundle format, such that OSGi bundles, like
>>> those on the Spring Source Repository could be utilised. It might also be a
>>> component in an eventual Container spec, if we can sort that out. Gregg has
>>> asked for container suggestions more than once, I'd like to see that
>>> conversation continued.
>>>
>>> If this solution was adopted, the dependency version metadata (Import
>>> Package) in the bundle would be noted, considered a minimum version as per
>>> the OSGi spec, but a later version would be utilized if it were found API
>>> compatible. However additional protection by static analysis would
>>> guarantee that a later version that wasn't API compatible would not be
>>> utilised. The reason I'm considering this compromise is that a bundle
>>> author cannot know in advance that later import packages will be compatible,
>>> hence Static Analysis can verify API compatibility. It is still possible
>>> for identical named classes to be loaded in the same jvm, in another
>>> classloader, this might occur if an earlier package version already exists
>>> but doesn't satisfy the Import Package version criteria; it wont be used as
>>> it is earlier than the minimum required version. The Classloader
>>> responsible for the bundle would keep a copy of the metadata available
>>> while it remains loaded, it would not utilise the OSGi interfaces to access
>>> the metadata. I envision Immutable Mirror Objects that can be transferred
>>> with marshalled instances and utilised for API compatibility checking. The
>>> Mirror objects would be created by a codebase service and stored in memory
>>> by the Classloader that download the class bytecode (River Bundles) being
>>> Mirrored, the mirror objects would contain the metadata from the bundle as
>>> well as the API, the mirror objects would be used by the classloader to
>>> determine which packages to export as well as check for import package
>>> compatibility, and by other Classloader's for compatibility. If no
>>> compatible packages are found loaded then the codebase services would be
>>> consulted, the import package mirror object would be submitted to the
>>> codebase services to locate a compatible package or bundle.
>>>
>>> In principle, if this is still objectionable, I'm happy to create an
>>> implementation utilising another container or bundle format that satisfies
>>> the requirements, provided that one is recommended. This list does have a
>>> history of heated debate regarding OSGi and Jini, I'd like to avoid any such
>>> unproductive arguments if possible.
>>>
>>> Regards,
>>>
>>> Peter.
>>>
>>>
>>> Niclas Hedhman wrote:
>>>
>>>> On Mon, Oct 19, 2009 at 7:40 AM, Peter Firmstone <ji...@zeus.net.au>
>>>> wrote:
>>>>
>>>>
>>>>>> Thanks Nic for your comments, pls see reply below.
>>>>>>
>>>>>>
>>>> I think we have diverged and discussing different things.
>>>>
>>>> Some time ago, it was said that the classloader of a class could be
>>>> retrieved by calling a static method. I have tried to point out that
>>>> this is fairly "impossible" in OSGi without collaboration with the
>>>> OSGi framework implementation, as multiple (even fully compatible, or
>>>> even exact same versions) instance of the same classes can be present
>>>> in different class spaces within the OSGi framework, and which package
>>>> got resolved to which bundle (and hence the classloader) is an
>>>> implementation detail in OSGi. There is no theory around that fact. I
>>>> am speaking of "locating a classloader of an existing class already
>>>> present", and not how to locate one externally to the JVM and OSGi
>>>> framework.
>>>>
>>>> See the difference?
>>>>
>>>> So if ClassA wants to load an implementation of InterfaceB, then that
>>>> implementation must be chosen from the class space that ClassA and
>>>> InterfaceB belongs to, and not only that it implements
>>>> InterfaceB+constraints.
>>>>
>>>>
>>>> Cheers
>>>>
>>>>
>>>
>>
>
>
Re: Cooperative Distributed computing - Versioning over time -
Container Metadata.
Posted by Calum Shaw-Mackay <ca...@gmail.com>.
This sounds very like Newton......
Another way might be to provide a new URL Handler and having a Version
ClassLoader higher up the classloader stack.
Although this doesn't really help in terms of attaching the version
required by a caller when trying to find a particular version, what
you might need is a decorated
classloader, that is built on the metadata supplied by the
service/client, with a version classloader acting as it's parent, so
when a class is requested by the service/client, it calls the
decorated classloader, which contains the metadata, which in turn
requests loadClass(name, version) from the Versioned ClassLoader.
Whichever way you want to tackle this, you'll need it to be as
transparent a mechanism as possible so that service/bundles/clients
that don;t specify versions work as they always have,
--Calum
2009/10/20 Peter Firmstone <ji...@zeus.net.au>:
> N.B.
>
> If we were able to utilise OSGi bundles, I can't then see why there couldn't
> be a way of setting up some kind of Bridge or Facade pattern to interoperate
> with OSGi. I'm only hypothesising, I don't have any particular
> implementation in mind. The advantage here would be the possibility of using
> common bundle code for both environments.
>
> Peter Firmstone wrote:
>>
>> Let me put it another way:
>>
>> I want to work toward a cooperative distributed computing platform using
>> Java & Jini / River, continuing from where Sun left, using lessons learned.
>>
>> 1. I'd like to set up a class loader framework building on
>> classworlds, to segregate incompatible class versions with the
>> same name, while allowing compatible implementations to
>> interoperate, based on Class loader visibility.
>> 2. I'd like to utilise the ASM bytecode library to perform static
>> analysis on bytecode to identify Package & Class API to determine
>> compatibility.
>> 3. I'd like to solve the lost code base problem using codebase
>> services combined with the suggested Class loading framework.
>> 4. I'd like to use codebase servcies to provide codebase proxy's for
>> often disconnected clients.
>> 5. I'd like to allow bytecode to evolve organically by taking
>> advantage of the flexibility of binary compatibility.
>> 6. I'd like some metadata to identify at the minimum ,Package
>> Versions, I could generate checksums based on Package API, however
>> this wouldn't identify which version is the latest, only which
>> Package implementations can substitute another's API where those
>> packages share the same name. Or identify packages with the same
>> name that are incompatible and segregate these in separate
>> Classloaders.
>> 7. If possible due to concerns about the performance impact of many
>> Classloaders, group some packages together when the author doesn't
>> intend those packages to evolve separately. Eg libraries.
>> 8. I'd like to make it possible to transfer class version and package
>> version information with serialized object instances for
>> compatible re-instantiation with the latest evolution of
>> compatible classes. All suggestions are welcome.
>>
>> Does anyone have any suggestions for suitable existing formats for
>> Containers with Version Metadata that might be utilised? If so what issues
>> are there to be aware of? What are the arguments for and Against?
>>
>> Note, I'm aware of the existing Java Version Spec that applies to
>> packages.
>>
>> I believe I made the mistake of suggesting a solution without properly
>> formulating or posing the problem, my apologies for any confusion this might
>> have caused, I should have raised a new thread / topic, In the original
>> thread, Gregg is looking at classloader dependency issues implementing a
>> plugin for Netbeans to make River more accessible to IDE developers, please
>> feel free to make suggestions, we need your help.
>>
>> I made a suggestion earlier to utilise the OSGi bundle metadata
>> specification, these are ordinary jar files with package version, exports
>> and dependency metadata. They are otherwise ordinary plain old jar
>> archives, quite useable as a jar file containing java class files that can
>> be utilised without the OSGi framework, which is what I was suggesting. I
>> attempted to imply that the exported packages could have their API
>> identified and shared in a compatible manner with other packages, using
>> Class loader isolation perhaps this is not a suitable solution due to the
>> confusion that it would cause with developers expecting an OSGi
>> implementation, utilising OSGi service interfaces. If this format was to be
>> utilised, it might have to be renamed, to River Codebase Bundles (for
>> uploading to codebases) whilst sharing the same format to avoid confusion.
>> It might be a subset of the OSGi bundle format, such that OSGi bundles, like
>> those on the Spring Source Repository could be utilised. It might also be a
>> component in an eventual Container spec, if we can sort that out. Gregg has
>> asked for container suggestions more than once, I'd like to see that
>> conversation continued.
>>
>> If this solution was adopted, the dependency version metadata (Import
>> Package) in the bundle would be noted, considered a minimum version as per
>> the OSGi spec, but a later version would be utilized if it were found API
>> compatible. However additional protection by static analysis would
>> guarantee that a later version that wasn't API compatible would not be
>> utilised. The reason I'm considering this compromise is that a bundle
>> author cannot know in advance that later import packages will be compatible,
>> hence Static Analysis can verify API compatibility. It is still possible
>> for identical named classes to be loaded in the same jvm, in another
>> classloader, this might occur if an earlier package version already exists
>> but doesn't satisfy the Import Package version criteria; it wont be used as
>> it is earlier than the minimum required version. The Classloader
>> responsible for the bundle would keep a copy of the metadata available
>> while it remains loaded, it would not utilise the OSGi interfaces to access
>> the metadata. I envision Immutable Mirror Objects that can be transferred
>> with marshalled instances and utilised for API compatibility checking. The
>> Mirror objects would be created by a codebase service and stored in memory
>> by the Classloader that download the class bytecode (River Bundles) being
>> Mirrored, the mirror objects would contain the metadata from the bundle as
>> well as the API, the mirror objects would be used by the classloader to
>> determine which packages to export as well as check for import package
>> compatibility, and by other Classloader's for compatibility. If no
>> compatible packages are found loaded then the codebase services would be
>> consulted, the import package mirror object would be submitted to the
>> codebase services to locate a compatible package or bundle.
>>
>> In principle, if this is still objectionable, I'm happy to create an
>> implementation utilising another container or bundle format that satisfies
>> the requirements, provided that one is recommended. This list does have a
>> history of heated debate regarding OSGi and Jini, I'd like to avoid any such
>> unproductive arguments if possible.
>>
>> Regards,
>>
>> Peter.
>>
>>
>> Niclas Hedhman wrote:
>>>
>>> On Mon, Oct 19, 2009 at 7:40 AM, Peter Firmstone <ji...@zeus.net.au>
>>> wrote:
>>>
>>>>>
>>>>> Thanks Nic for your comments, pls see reply below.
>>>>>
>>>
>>> I think we have diverged and discussing different things.
>>>
>>> Some time ago, it was said that the classloader of a class could be
>>> retrieved by calling a static method. I have tried to point out that
>>> this is fairly "impossible" in OSGi without collaboration with the
>>> OSGi framework implementation, as multiple (even fully compatible, or
>>> even exact same versions) instance of the same classes can be present
>>> in different class spaces within the OSGi framework, and which package
>>> got resolved to which bundle (and hence the classloader) is an
>>> implementation detail in OSGi. There is no theory around that fact. I
>>> am speaking of "locating a classloader of an existing class already
>>> present", and not how to locate one externally to the JVM and OSGi
>>> framework.
>>>
>>> See the difference?
>>>
>>> So if ClassA wants to load an implementation of InterfaceB, then that
>>> implementation must be chosen from the class space that ClassA and
>>> InterfaceB belongs to, and not only that it implements
>>> InterfaceB+constraints.
>>>
>>>
>>> Cheers
>>>
>>
>>
>
>
Re: Cooperative Distributed computing - Versioning over time - Container
Metadata.
Posted by Peter Firmstone <ji...@zeus.net.au>.
N.B.
If we were able to utilise OSGi bundles, I can't then see why there
couldn't be a way of setting up some kind of Bridge or Facade pattern to
interoperate with OSGi. I'm only hypothesising, I don't have any
particular implementation in mind. The advantage here would be the
possibility of using common bundle code for both environments.
Peter Firmstone wrote:
> Let me put it another way:
>
> I want to work toward a cooperative distributed computing platform
> using Java & Jini / River, continuing from where Sun left, using
> lessons learned.
>
> 1. I'd like to set up a class loader framework building on
> classworlds, to segregate incompatible class versions with the
> same name, while allowing compatible implementations to
> interoperate, based on Class loader visibility.
> 2. I'd like to utilise the ASM bytecode library to perform static
> analysis on bytecode to identify Package & Class API to determine
> compatibility.
> 3. I'd like to solve the lost code base problem using codebase
> services combined with the suggested Class loading framework.
> 4. I'd like to use codebase servcies to provide codebase proxy's for
> often disconnected clients.
> 5. I'd like to allow bytecode to evolve organically by taking
> advantage of the flexibility of binary compatibility.
> 6. I'd like some metadata to identify at the minimum ,Package
> Versions, I could generate checksums based on Package API, however
> this wouldn't identify which version is the latest, only which
> Package implementations can substitute another's API where those
> packages share the same name. Or identify packages with the same
> name that are incompatible and segregate these in separate
> Classloaders.
> 7. If possible due to concerns about the performance impact of many
> Classloaders, group some packages together when the author doesn't
> intend those packages to evolve separately. Eg libraries.
> 8. I'd like to make it possible to transfer class version and package
> version information with serialized object instances for
> compatible re-instantiation with the latest evolution of
> compatible classes. All suggestions are welcome.
>
> Does anyone have any suggestions for suitable existing formats for
> Containers with Version Metadata that might be utilised?
> If so what issues are there to be aware of? What are the arguments
> for and Against?
>
> Note, I'm aware of the existing Java Version Spec that applies to
> packages.
>
> I believe I made the mistake of suggesting a solution without properly
> formulating or posing the problem, my apologies for any confusion this
> might have caused, I should have raised a new thread / topic, In the
> original thread, Gregg is looking at classloader dependency issues
> implementing a plugin for Netbeans to make River more accessible to
> IDE developers, please feel free to make suggestions, we need your help.
>
> I made a suggestion earlier to utilise the OSGi bundle metadata
> specification, these are ordinary jar files with package version,
> exports and dependency metadata. They are otherwise ordinary plain
> old jar archives, quite useable as a jar file containing java class
> files that can be utilised without the OSGi framework, which is what
> I was suggesting. I attempted to imply that the exported packages
> could have their API identified and shared in a compatible manner with
> other packages, using Class loader isolation perhaps this is not a
> suitable solution due to the confusion that it would cause with
> developers expecting an OSGi implementation, utilising OSGi service
> interfaces. If this format was to be utilised, it might have to be
> renamed, to River Codebase Bundles (for uploading to codebases) whilst
> sharing the same format to avoid confusion. It might be a subset of
> the OSGi bundle format, such that OSGi bundles, like those on the
> Spring Source Repository could be utilised. It might also be a
> component in an eventual Container spec, if we can sort that out.
> Gregg has asked for container suggestions more than once, I'd like to
> see that conversation continued.
>
> If this solution was adopted, the dependency version metadata (Import
> Package) in the bundle would be noted, considered a minimum version as
> per the OSGi spec, but a later version would be utilized if it were
> found API compatible. However additional protection by static
> analysis would guarantee that a later version that wasn't API
> compatible would not be utilised. The reason I'm considering this
> compromise is that a bundle author cannot know in advance that later
> import packages will be compatible, hence Static Analysis can verify
> API compatibility. It is still possible for identical named classes
> to be loaded in the same jvm, in another classloader, this might occur
> if an earlier package version already exists but doesn't satisfy the
> Import Package version criteria; it wont be used as it is earlier than
> the minimum required version. The Classloader responsible for the
> bundle would keep a copy of the metadata available while it remains
> loaded, it would not utilise the OSGi interfaces to access the
> metadata. I envision Immutable Mirror Objects that can be transferred
> with marshalled instances and utilised for API compatibility
> checking. The Mirror objects would be created by a codebase service
> and stored in memory by the Classloader that download the class
> bytecode (River Bundles) being Mirrored, the mirror objects would
> contain the metadata from the bundle as well as the API, the mirror
> objects would be used by the classloader to determine which packages
> to export as well as check for import package compatibility, and by
> other Classloader's for compatibility. If no compatible packages are
> found loaded then the codebase services would be consulted, the import
> package mirror object would be submitted to the codebase services to
> locate a compatible package or bundle.
>
> In principle, if this is still objectionable, I'm happy to create an
> implementation utilising another container or bundle format that
> satisfies the requirements, provided that one is recommended. This
> list does have a history of heated debate regarding OSGi and Jini, I'd
> like to avoid any such unproductive arguments if possible.
>
> Regards,
>
> Peter.
>
>
> Niclas Hedhman wrote:
>> On Mon, Oct 19, 2009 at 7:40 AM, Peter Firmstone <ji...@zeus.net.au>
>> wrote:
>>
>>>> Thanks Nic for your comments, pls see reply below.
>>>>
>>
>> I think we have diverged and discussing different things.
>>
>> Some time ago, it was said that the classloader of a class could be
>> retrieved by calling a static method. I have tried to point out that
>> this is fairly "impossible" in OSGi without collaboration with the
>> OSGi framework implementation, as multiple (even fully compatible, or
>> even exact same versions) instance of the same classes can be present
>> in different class spaces within the OSGi framework, and which package
>> got resolved to which bundle (and hence the classloader) is an
>> implementation detail in OSGi. There is no theory around that fact. I
>> am speaking of "locating a classloader of an existing class already
>> present", and not how to locate one externally to the JVM and OSGi
>> framework.
>>
>> See the difference?
>>
>> So if ClassA wants to load an implementation of InterfaceB, then that
>> implementation must be chosen from the class space that ClassA and
>> InterfaceB belongs to, and not only that it implements
>> InterfaceB+constraints.
>>
>>
>> Cheers
>>
>
>
Cooperative Distributed computing - Versioning over time - Container
Metadata.
Posted by Peter Firmstone <ji...@zeus.net.au>.
Let me put it another way:
I want to work toward a cooperative distributed computing platform using
Java & Jini / River, continuing from where Sun left, using lessons learned.
1. I'd like to set up a class loader framework building on
classworlds, to segregate incompatible class versions with the
same name, while allowing compatible implementations to
interoperate, based on Class loader visibility.
2. I'd like to utilise the ASM bytecode library to perform static
analysis on bytecode to identify Package & Class API to determine
compatibility.
3. I'd like to solve the lost code base problem using codebase
services combined with the suggested Class loading framework.
4. I'd like to use codebase servcies to provide codebase proxy's for
often disconnected clients.
5. I'd like to allow bytecode to evolve organically by taking
advantage of the flexibility of binary compatibility.
6. I'd like some metadata to identify at the minimum ,Package
Versions, I could generate checksums based on Package API, however
this wouldn't identify which version is the latest, only which
Package implementations can substitute another's API where those
packages share the same name. Or identify packages with the same
name that are incompatible and segregate these in separate
Classloaders.
7. If possible due to concerns about the performance impact of many
Classloaders, group some packages together when the author doesn't
intend those packages to evolve separately. Eg libraries.
8. I'd like to make it possible to transfer class version and package
version information with serialized object instances for
compatible re-instantiation with the latest evolution of
compatible classes. All suggestions are welcome.
Does anyone have any suggestions for suitable existing formats for
Containers with Version Metadata that might be utilised?
If so what issues are there to be aware of? What are the arguments for
and Against?
Note, I'm aware of the existing Java Version Spec that applies to packages.
I believe I made the mistake of suggesting a solution without properly
formulating or posing the problem, my apologies for any confusion this
might have caused, I should have raised a new thread / topic, In the
original thread, Gregg is looking at classloader dependency issues
implementing a plugin for Netbeans to make River more accessible to IDE
developers, please feel free to make suggestions, we need your help.
I made a suggestion earlier to utilise the OSGi bundle metadata
specification, these are ordinary jar files with package version,
exports and dependency metadata. They are otherwise ordinary plain old
jar archives, quite useable as a jar file containing java class files
that can be utilised without the OSGi framework, which is what I was
suggesting. I attempted to imply that the exported packages could have
their API identified and shared in a compatible manner with other
packages, using Class loader isolation perhaps this is not a suitable
solution due to the confusion that it would cause with developers
expecting an OSGi implementation, utilising OSGi service interfaces. If
this format was to be utilised, it might have to be renamed, to River
Codebase Bundles (for uploading to codebases) whilst sharing the same
format to avoid confusion. It might be a subset of the OSGi bundle
format, such that OSGi bundles, like those on the Spring Source
Repository could be utilised. It might also be a component in an
eventual Container spec, if we can sort that out. Gregg has asked for
container suggestions more than once, I'd like to see that conversation
continued.
If this solution was adopted, the dependency version metadata (Import
Package) in the bundle would be noted, considered a minimum version as
per the OSGi spec, but a later version would be utilized if it were
found API compatible. However additional protection by static analysis
would guarantee that a later version that wasn't API compatible would
not be utilised. The reason I'm considering this compromise is that a
bundle author cannot know in advance that later import packages will be
compatible, hence Static Analysis can verify API compatibility. It is
still possible for identical named classes to be loaded in the same jvm,
in another classloader, this might occur if an earlier package version
already exists but doesn't satisfy the Import Package version criteria;
it wont be used as it is earlier than the minimum required version. The
Classloader responsible for the bundle would keep a copy of the
metadata available while it remains loaded, it would not utilise the
OSGi interfaces to access the metadata. I envision Immutable Mirror
Objects that can be transferred with marshalled instances and utilised
for API compatibility checking. The Mirror objects would be created by
a codebase service and stored in memory by the Classloader that download
the class bytecode (River Bundles) being Mirrored, the mirror objects
would contain the metadata from the bundle as well as the API, the
mirror objects would be used by the classloader to determine which
packages to export as well as check for import package compatibility,
and by other Classloader's for compatibility. If no compatible packages
are found loaded then the codebase services would be consulted, the
import package mirror object would be submitted to the codebase services
to locate a compatible package or bundle.
In principle, if this is still objectionable, I'm happy to create an
implementation utilising another container or bundle format that
satisfies the requirements, provided that one is recommended. This list
does have a history of heated debate regarding OSGi and Jini, I'd like
to avoid any such unproductive arguments if possible.
Regards,
Peter.
Niclas Hedhman wrote:
> On Mon, Oct 19, 2009 at 7:40 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>>> Thanks Nic for your comments, pls see reply below.
>>>
>
> I think we have diverged and discussing different things.
>
> Some time ago, it was said that the classloader of a class could be
> retrieved by calling a static method. I have tried to point out that
> this is fairly "impossible" in OSGi without collaboration with the
> OSGi framework implementation, as multiple (even fully compatible, or
> even exact same versions) instance of the same classes can be present
> in different class spaces within the OSGi framework, and which package
> got resolved to which bundle (and hence the classloader) is an
> implementation detail in OSGi. There is no theory around that fact. I
> am speaking of "locating a classloader of an existing class already
> present", and not how to locate one externally to the JVM and OSGi
> framework.
>
> See the difference?
>
> So if ClassA wants to load an implementation of InterfaceB, then that
> implementation must be chosen from the class space that ClassA and
> InterfaceB belongs to, and not only that it implements
> InterfaceB+constraints.
>
>
> Cheers
>
Re: Separating out RMIClassLoader dependencies
Posted by Peter Firmstone <ji...@zeus.net.au>.
> Niclas Hedhman wrote:
>> Some time ago, it was said that the classloader of a class could be
>> retrieved by calling a static method. I have tried to point out that
>> this is fairly "impossible" in OSGi without collaboration with the
>> OSGi framework implementation, as multiple (even fully compatible, or
>> even exact same versions) instance of the same classes can be present
>> in different class spaces within the OSGi framework, and which package
>> got resolved to which bundle (and hence the classloader) is an
>> implementation detail in OSGi. There is no theory around that fact. I
>> am speaking of "locating a classloader of an existing class already
>> present", and not how to locate one externally to the JVM and OSGi
>> framework.
It would be interesting to see how Paremus have implemented their Jini,
OSGi integration, all information is certainly welcome.
Re: Separating out RMIClassLoader dependencies
Posted by Peter Firmstone <ji...@zeus.net.au>.
This looks like a fine idea, so we can have several pluggable
ClassLoader Architecture options, sounds very interesting.
I need to think about how I might also utilise this pluggable
architecture before I get back to you. See if I can achieve the
required functionality with the existing interface, or whether I'd need
to add a method.
Cheers,
Peter.
Gregg Wonderly wrote:
>
> The primary issue that my previous posting was trying to sort out, was
> the fact that inside of the core of River class loading, is the use of
> several methods on the RMIClassLoader class, which then results in
> calls to the configured and loaded instance of the RMIClassLoaderSpi
> implementation. In containers and IDE environments, where
> classloading is customized and managed very differently than what JSE
> does by itself, getting an RMIClassLoaderSpi instance to be used
> through the normal means, can be nearly impossible.
>
> So, I am investigating not using the RMIClassLoaderSpi mechanisms that
> are accessed through RMIClassLoader.
>
> Instead, I have an interface defined (this is what I have so far) as
> follows:
>
> public interface CodebaseClassAccess {
> public Class loadClass(String codebase, String name )
> throws IOException, ClassNotFoundException;
> public Class loadClass(String codebase, String name,
> ClassLoader defaultLoader )
> throws IOException,ClassNotFoundException;
> public Class loadProxyClass(String codebase, String[] interfaceNames,
> ClassLoader defaultLoader )
> throws IOException,ClassNotFoundException;
> public String getClassAnnotation( Class cls );
> public ClassLoader getClassLoader(String codebase) throws
> IOException;
> public ClassLoader createClassLoader( URL[] urls,
> ClassLoader parent,
> boolean requireDlPerm,
> AccessControlContext ctx );
> public ClassLoader getParentContextClassLoader();
> public ClassLoader getSystemContextClassLoader( ClassLoader
> defaultLoader );
> }
>
> This is an SPI interface. There is a class with static methods that
> mirrors this interface. That class has a setter on it to set the
> implementation of this
> interface to use. Without a call to the setter, the default is the
> use of RMIClassLoaderSpi.
>
> I have an implementation that I am using in my netbeans development
> that is shown below.
>
> There are three important issues that this covers.
>
> 1) It covers the fact that the use of null as a "default" parent
> classloader is not going to work in many cases, because the jini/river
> jars are likely not in the JVM system class loader.
> 2) My changes for allowing "never-preferred" classes to prohibit
> downloading
> jars until services will be activated, need to know what loader to
> use for
> non-preferred classes since it is not usually going to be the JVM
> system
> class loader.
> 3) Class.forName( name ) will only use the preferred class loader or
> system
> class loader and in some cases, there is more control needed.
>
> In the end, I am making changes that start to allow other class
> loading/packaging/platforming to be plugged into the
> PreferredClassLoader. I'm doing this, because I think that if we want
> Jini to work in other places, we have to be able to also honor the
> ability to plugin Jini services that need preferred classes to work as
> part of their bug fixing strategy.
>
> I've got some stuff working now, but there is still more work to do to
> provide a useful package to try out.
>
> Gregg Wonderly
>
> public class NetbeansCodebaseClassLoaderAccess
> implements CodebaseClassAccess {
> public static PreferredClassProvider provider = new
> PreferredClassProvider();
>
> public @Override ClassLoader getParentContextClassLoader() {
> return Lookup.getDefault().lookup(ClassLoader.class);
> }
>
> public @Override Class loadClass( String codebase,
> String name,
> ClassLoader defaultLoader ) throws IOException,
> ClassNotFoundException {
> return provider.loadClass( codebase, name, defaultLoader );
> }
>
> public @Override Class loadProxyClass(String codebase, String[]
> interfaceNames,
> ClassLoader defaultLoader) throws IOException,
> ClassNotFoundException {
> return provider.loadProxyClass( codebase, interfaceNames,
> defaultLoader );
> }
>
> public @Override String getClassAnnotation( Class cls ) {
> return provider.getClassAnnotation( cls );
> }
>
> public @Override ClassLoader getClassLoader(String codebase)
> throws IOException {
> return provider.getClassLoader( codebase );
> }
>
> public @Override Class loadClass(String codebase, String name)
> throws IOException, ClassNotFoundException {
> return provider.loadClass( codebase, name,
> Thread.currentThread().getContextClassLoader() );
> }
>
> public @Override ClassLoader createClassLoader( final URL[] urls,
> final ClassLoader parent,
> final boolean requireDlPerm, final AccessControlContext
> ctx) {
> return AccessController.doPrivileged(new
> PrivilegedAction<ClassLoader>() {
> public ClassLoader run() {
> return new PreferredClassLoader(urls, parent,
> null, requireDlPerm);
> }
> }, ctx );
> }
>
> public ClassLoader getSystemContextClassLoader( ClassLoader
> defaultLoader ) {
> // defaultLoader will be the PreferredClassLoader instance.
> We need to just
> // return that loader instead of the
> defaultLoader.getClass().getClassLoader()
> // business that usually occurs. That business would change
> the class loading
> // scope to completely remove all view of the system and the
> codebase classes.
> return defaultLoader != null ? defaultLoader :
> Lookup.getDefault().lookup(ClassLoader.class);
> }
> }
>
>
Re: Separating out RMIClassLoader dependencies
Posted by Gregg Wonderly <gr...@wonderly.org>.
Niclas Hedhman wrote:
> Some time ago, it was said that the classloader of a class could be
> retrieved by calling a static method. I have tried to point out that
> this is fairly "impossible" in OSGi without collaboration with the
> OSGi framework implementation, as multiple (even fully compatible, or
> even exact same versions) instance of the same classes can be present
> in different class spaces within the OSGi framework, and which package
> got resolved to which bundle (and hence the classloader) is an
> implementation detail in OSGi. There is no theory around that fact. I
> am speaking of "locating a classloader of an existing class already
> present", and not how to locate one externally to the JVM and OSGi
> framework.
The primary issue that my previous posting was trying to sort out, was the fact
that inside of the core of River class loading, is the use of several methods on
the RMIClassLoader class, which then results in calls to the configured and
loaded instance of the RMIClassLoaderSpi implementation. In containers and IDE
environments, where classloading is customized and managed very differently than
what JSE does by itself, getting an RMIClassLoaderSpi instance to be used
through the normal means, can be nearly impossible.
So, I am investigating not using the RMIClassLoaderSpi mechanisms that are
accessed through RMIClassLoader.
Instead, I have an interface defined (this is what I have so far) as follows:
public interface CodebaseClassAccess {
public Class loadClass(String codebase, String name )
throws IOException, ClassNotFoundException;
public Class loadClass(String codebase, String name,
ClassLoader defaultLoader )
throws IOException,ClassNotFoundException;
public Class loadProxyClass(String codebase, String[] interfaceNames,
ClassLoader defaultLoader )
throws IOException,ClassNotFoundException;
public String getClassAnnotation( Class cls );
public ClassLoader getClassLoader(String codebase) throws IOException;
public ClassLoader createClassLoader( URL[] urls,
ClassLoader parent,
boolean requireDlPerm,
AccessControlContext ctx );
public ClassLoader getParentContextClassLoader();
public ClassLoader getSystemContextClassLoader( ClassLoader defaultLoader );
}
This is an SPI interface. There is a class with static methods that mirrors
this interface. That class has a setter on it to set the implementation of this
interface to use. Without a call to the setter, the default is the use of
RMIClassLoaderSpi.
I have an implementation that I am using in my netbeans development that is
shown below.
There are three important issues that this covers.
1) It covers the fact that the use of null as a "default" parent classloader is
not going to work in many cases, because the jini/river jars are likely not in
the JVM system class loader.
2) My changes for allowing "never-preferred" classes to prohibit downloading
jars until services will be activated, need to know what loader to use for
non-preferred classes since it is not usually going to be the JVM system
class loader.
3) Class.forName( name ) will only use the preferred class loader or system
class loader and in some cases, there is more control needed.
In the end, I am making changes that start to allow other class
loading/packaging/platforming to be plugged into the PreferredClassLoader. I'm
doing this, because I think that if we want Jini to work in other places, we
have to be able to also honor the ability to plugin Jini services that need
preferred classes to work as part of their bug fixing strategy.
I've got some stuff working now, but there is still more work to do to provide a
useful package to try out.
Gregg Wonderly
public class NetbeansCodebaseClassLoaderAccess
implements CodebaseClassAccess {
public static PreferredClassProvider provider = new PreferredClassProvider();
public @Override ClassLoader getParentContextClassLoader() {
return Lookup.getDefault().lookup(ClassLoader.class);
}
public @Override Class loadClass( String codebase,
String name,
ClassLoader defaultLoader ) throws IOException,
ClassNotFoundException {
return provider.loadClass( codebase, name, defaultLoader );
}
public @Override Class loadProxyClass(String codebase, String[] interfaceNames,
ClassLoader defaultLoader) throws IOException,
ClassNotFoundException {
return provider.loadProxyClass( codebase, interfaceNames, defaultLoader );
}
public @Override String getClassAnnotation( Class cls ) {
return provider.getClassAnnotation( cls );
}
public @Override ClassLoader getClassLoader(String codebase) throws
IOException {
return provider.getClassLoader( codebase );
}
public @Override Class loadClass(String codebase, String name)
throws IOException, ClassNotFoundException {
return provider.loadClass( codebase, name,
Thread.currentThread().getContextClassLoader() );
}
public @Override ClassLoader createClassLoader( final URL[] urls, final
ClassLoader parent,
final boolean requireDlPerm, final AccessControlContext ctx) {
return AccessController.doPrivileged(new PrivilegedAction<ClassLoader>() {
public ClassLoader run() {
return new PreferredClassLoader(urls, parent, null,
requireDlPerm);
}
}, ctx );
}
public ClassLoader getSystemContextClassLoader( ClassLoader defaultLoader ) {
// defaultLoader will be the PreferredClassLoader instance. We need to
just
// return that loader instead of the
defaultLoader.getClass().getClassLoader()
// business that usually occurs. That business would change the class
loading
// scope to completely remove all view of the system and the codebase
classes.
return defaultLoader != null ? defaultLoader :
Lookup.getDefault().lookup(ClassLoader.class);
}
}
Re: Separating out RMIClassLoader dependencies
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Mon, Oct 19, 2009 at 7:40 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
>> Thanks Nic for your comments, pls see reply below.
I think we have diverged and discussing different things.
Some time ago, it was said that the classloader of a class could be
retrieved by calling a static method. I have tried to point out that
this is fairly "impossible" in OSGi without collaboration with the
OSGi framework implementation, as multiple (even fully compatible, or
even exact same versions) instance of the same classes can be present
in different class spaces within the OSGi framework, and which package
got resolved to which bundle (and hence the classloader) is an
implementation detail in OSGi. There is no theory around that fact. I
am speaking of "locating a classloader of an existing class already
present", and not how to locate one externally to the JVM and OSGi
framework.
See the difference?
So if ClassA wants to load an implementation of InterfaceB, then that
implementation must be chosen from the class space that ClassA and
InterfaceB belongs to, and not only that it implements
InterfaceB+constraints.
Cheers
--
Niclas Hedhman, Software Developer
http://www.qi4j.org - New Energy for Java
I live here; http://tinyurl.com/2qq9er
I work here; http://tinyurl.com/2ymelc
I relax here; http://tinyurl.com/2cgsug
Re: Separating out RMIClassLoader dependencies
Posted by Peter Firmstone <ji...@zeus.net.au>.
Ignore this message, it was sent during drafting. The real message will
follow.
Peter Firmstone wrote:
> Thanks Nic for your comments, pls see reply below.
>
> Niclas Hedhman wrote:
>> A big chunk of your bundle analysis already exist in a widely used OSGi
>> tool, named BND. Most OSGi developers use it, since it is near
>> impossible to
>> create the metadata by hand, especially the "uses" constraint.
>>
> Definitely agree, quite useful for eliminating syntax errors.
>> But before that we spoke of figuring out how a particular implementation
>> wire up a random set of bundles. That is not predictable, nor does
>> the OSGi
>> spec provides an API to find out. Well, kind of... If you have the
>> loaded
>> class, you can ask from which bundle it came, but unfortunately the
>> StackElement doesn't give you the Class, only the class name, which
>> isn't
>> enough.
>>
> Bytecode analysis, can identify all method signatures, all fields, the
> level of access, public, package private, protected or private.
>
> It can also identify the serialVersionUID for classes that implement
> Serializable.
>
> In addition if a bundle is used, the Metadata in the bundle indicates
> packages to be exported, which defines the publicly accessible API.
>
> This information can be stored in Static Mirror Objects, for each
> bundle, package and class.
>
> It is also possible to support the Java Product Versioning
> Specification, the difference being that there would be one
> Classloader per package, instead of per bundle. Public and Protected
> Public Class API signatures and serialVersionUID's stored in class
> mirror objects, would determine API compatibility. Preference would
> be given to compatible Package Implementations that had greater
> version ID's.
>
> Since the Static Bytecode Anaysis would allow identification of API
> signatures, most importantly a Packages Public API, Protected API and
> Class serialVersionUID's, it allows comparison of API between
> different package versions. - Package API.
>
> Special consideration has to be given to Package API that is shared
> between multiple packages, lazy loading requires some thought too. I
> think all dependencies will have to be resolved prior to loading for
> all packages. Failed resolution would require a JVM restart, with
> state persistence.
>
> It's possible to additionally support the Java Product Versioning
> Specification, where maximum sharing of classes is desired, eg
> globally visible classes, as this increases the number of classloaders
> and reduces the probability that a Package API conflict cannot be
> handled gracefully, considering the size and availability of OSGi
> bundle repository's, the likelyhood isn't great, but it may be useful
> where new software is undergoing rapid development and API revision,
> where shared classes need to be persisted.
>
> Perhaps I should name the Mirror Interface, in Reflective Terms,
> StaticMirror - to indicate it doesn't change, with implementations
> such as ClassStaticMirror, PackageStaticMirror and BundleStaticMirror,
> Mirror objects produced by analysis could be stored in a hash that
> looks up packages by name.
>
> I was also thinking that for dependency resolution, I could have a
> PackageDependencyMirror Object, which ClassLoaders could submit to a
> codebase service for resolution of the latest compatible packages.
> The required packages
>
>
>
>
>> But I agree with you that with Jini tech and OSGi's well-defined
>> classloading semantics one can do really cool stuff. Including
>> getting rid
>> of some of the more troubling Jini aspects (codebase).
>>
>> -- Niclas
>>
>> On Oct 16, 2009 8:21 PM, "Peter Firmstone" <ji...@zeus.net.au> wrote:
>>
>> Hmm, that sounds like opportunity.
>>
>> A dedicated codebase service has all the time in the world to burn
>> processor
>> cycles on analysis, as its main task is simply serving up jar files over
>> networks.
>>
>> Bytecode analysis, identifies class, package and possibly module API,
>> which
>> can be stored with the harvested metadata in mirror objects, one for
>> each
>> class, package and module. Optional Package metadata could be a
>> potential
>> source of information too.
>>
>> Bundles, depend upon and export packages. Once the API is identified
>> using
>> bytecode analysis, fast comparison using mirror objects could identify
>> compatibility.
>>
>> This could be utilised in two ways:
>>
>> 1. As a check of backward compatibility for modules / packages over
>> different release versions, for substituting later compatible bundle
>> versions, if desired.
>> 2. When a dependency on a package only utilises a subset of that
>> package,
>> the actual API requirements may still be satisfied, even though
>> different
>> release versions of that package may not usually be interchangeable
>> or fully
>> compatible.
>>
>> A module or bundle would exist within its own classloader, in the
>> local jvm.
>> The packages or modules upon which it depends could be made
>> available from
>> other classloaders.
>>
>> Bundles could be uploaded to codebase services or a codebase service
>> could
>> retrieve the bundles from designated repository's, perform analysis,
>> then
>> make the bundles available in a location independent manner, to prevent
>> codebase loss and allow for redundant codebase services.
>>
>> Then all one need do is to upload application bundles to the codebase
>> server
>> and register a service, the service and bytecode could be provided
>> independently, the codebase service can provide entire application
>> bytecode
>> built on Jini services and other third party libraries.
>>
>> Have a look on springsource, there are many OSGi jar bundles available,
>> these are simply jar files with Metadata.
>>
>> Someone's done so much hard work already, why not ride the wave?
>> There are
>> already support tools available to create application bundle manifests.
>>
>> The reason I'm considering bundles, is it reduces the number of
>> classloaders
>> required, one per bundle as opposed to one per package. One cannot
>> rely on
>> standard java Package meta data to exist in jar files.
>>
>> ********This doesn't require an OSGi framework.*********
>>
>> http://blog.springsource.com/2008/02/18/creating-osgi-bundles/
>>
>> http://www.springsource.com/repository/app/bundle?query=A
>>
>> http://www.springsource.com/repository/app/faq
>>
>> >From the website:
>>
>>
>> What is the SpringSource Enterprise Bundle Repository?
>>
>> The SpringSource Enterprise Bundle Repository is a collection of open
>> source
>> libraries commonly used for developing enterprise Java applications
>> with the
>> Spring Framework. The repository contains jar files (bundles) and
>> library
>> definition (".libd") files. A library defines a collection of bundles
>> that
>> are often used together for some purpose (e.g. the "Spring Framework"
>> library). There are hundreds of bundles contained in the repository.
>>
>> The repository meets the following criteria:
>>
>> * Every jar file in the repository is a valid OSGi bundle. Any jar
>> you download from the repository can be deployed as-is into an
>> OSGi Service Platform and the SpringSource dm Server. It can also
>> be used as a regular jar file outside of OSGi.
>> * Every bundle and library has full version information associated
>> with it. The package export information for a bundle contains
>> version information, and the package import information for a
>> bundle contains full version range compatibility information.
>> * The repository is transitively complete. The mandatory
>> dependencies of any bundle are guaranteed to also be in the
>> repository. Most of the optional dependencies of any bundle in the
>> repository will also be present. The bundles listed in any library
>> definition are guaranteed to be in the repository.
>> * The repository is self-consistent. Before any artefact is uploaded
>> to the repository, we verify that it can be installed, resolved,
>> and started in an OSGi Service Platform (using the same profile as
>> the SpringSource dm Server) alongside all of the other bundles in
>> the repository.
>> * The repository can be used from Ivy and Maven based builds.
>>
>> Cheers,
>>
>> Peter.
>>
>> Niclas Hedhman wrote: > > On Fri, Oct 16, 2009 at 1:04 PM, Niclas
>> Hedhman <
>> niclas@hedhman.org> wro...
>>
>>
>
>
Re: Separating out RMIClassLoader dependencies
Posted by Peter Firmstone <ji...@zeus.net.au>.
Thanks Nic for your comments, pls see reply below.
Niclas Hedhman wrote:
> A big chunk of your bundle analysis already exist in a widely used OSGi
> tool, named BND. Most OSGi developers use it, since it is near impossible to
> create the metadata by hand, especially the "uses" constraint.
>
Definitely agree, quite useful for eliminating syntax errors.
> But before that we spoke of figuring out how a particular implementation
> wire up a random set of bundles. That is not predictable, nor does the OSGi
> spec provides an API to find out. Well, kind of... If you have the loaded
> class, you can ask from which bundle it came, but unfortunately the
> StackElement doesn't give you the Class, only the class name, which isn't
> enough.
>
Bytecode analysis, can identify all method signatures, all fields, the
level of access, public, package private, protected or private.
It can also identify the serialVersionUID for classes that implement
Serializable.
In addition if a bundle is used, the Metadata in the bundle indicates
packages to be exported, which defines the publicly accessible API.
This information can be stored in Static Mirror Objects, for each
bundle, package and class.
It is also possible to support the Java Product Versioning
Specification, the difference being that there would be one Classloader
per package, instead of per bundle. Public and Protected Public Class
API signatures and serialVersionUID's stored in class mirror objects,
would determine API compatibility. Preference would be given to
compatible Package Implementations that had greater version ID's.
Since the Static Bytecode Anaysis would allow identification of API
signatures, most importantly a Packages Public API, Protected API and
Class serialVersionUID's, it allows comparison of API between different
package versions. - Package API.
Special consideration has to be given to Package API that is shared
between multiple packages, lazy loading requires some thought too. I
think all dependencies will have to be resolved prior to loading for all
packages. Failed resolution would require a JVM restart, with state
persistence.
It's possible to additionally support the Java Product Versioning
Specification, where maximum sharing of classes is desired, eg globally
visible classes, as this increases the number of classloaders and
reduces the probability that a Package API conflict cannot be handled
gracefully, considering the size and availability of OSGi bundle
repository's, the likelyhood isn't great, but it may be useful where new
software is undergoing rapid development and API revision, where shared
classes need to be persisted.
Perhaps I should name the Mirror Interface, in Reflective Terms,
StaticMirror - to indicate it doesn't change, with implementations such
as ClassStaticMirror, PackageStaticMirror and BundleStaticMirror, Mirror
objects produced by analysis could be stored in a hash that looks up
packages by name.
I was also thinking that for dependency resolution, I could have a
PackageDependencyMirror Object, which ClassLoaders could submit to a
codebase service for resolution of the latest compatible packages. The
required packages
> But I agree with you that with Jini tech and OSGi's well-defined
> classloading semantics one can do really cool stuff. Including getting rid
> of some of the more troubling Jini aspects (codebase).
>
> -- Niclas
>
> On Oct 16, 2009 8:21 PM, "Peter Firmstone" <ji...@zeus.net.au> wrote:
>
> Hmm, that sounds like opportunity.
>
> A dedicated codebase service has all the time in the world to burn processor
> cycles on analysis, as its main task is simply serving up jar files over
> networks.
>
> Bytecode analysis, identifies class, package and possibly module API, which
> can be stored with the harvested metadata in mirror objects, one for each
> class, package and module. Optional Package metadata could be a potential
> source of information too.
>
> Bundles, depend upon and export packages. Once the API is identified using
> bytecode analysis, fast comparison using mirror objects could identify
> compatibility.
>
> This could be utilised in two ways:
>
> 1. As a check of backward compatibility for modules / packages over
> different release versions, for substituting later compatible bundle
> versions, if desired.
> 2. When a dependency on a package only utilises a subset of that package,
> the actual API requirements may still be satisfied, even though different
> release versions of that package may not usually be interchangeable or fully
> compatible.
>
> A module or bundle would exist within its own classloader, in the local jvm.
> The packages or modules upon which it depends could be made available from
> other classloaders.
>
> Bundles could be uploaded to codebase services or a codebase service could
> retrieve the bundles from designated repository's, perform analysis, then
> make the bundles available in a location independent manner, to prevent
> codebase loss and allow for redundant codebase services.
>
> Then all one need do is to upload application bundles to the codebase server
> and register a service, the service and bytecode could be provided
> independently, the codebase service can provide entire application bytecode
> built on Jini services and other third party libraries.
>
> Have a look on springsource, there are many OSGi jar bundles available,
> these are simply jar files with Metadata.
>
> Someone's done so much hard work already, why not ride the wave? There are
> already support tools available to create application bundle manifests.
>
> The reason I'm considering bundles, is it reduces the number of classloaders
> required, one per bundle as opposed to one per package. One cannot rely on
> standard java Package meta data to exist in jar files.
>
> ********This doesn't require an OSGi framework.*********
>
> http://blog.springsource.com/2008/02/18/creating-osgi-bundles/
>
> http://www.springsource.com/repository/app/bundle?query=A
>
> http://www.springsource.com/repository/app/faq
>
> >From the website:
>
>
> What is the SpringSource Enterprise Bundle Repository?
>
> The SpringSource Enterprise Bundle Repository is a collection of open source
> libraries commonly used for developing enterprise Java applications with the
> Spring Framework. The repository contains jar files (bundles) and library
> definition (".libd") files. A library defines a collection of bundles that
> are often used together for some purpose (e.g. the "Spring Framework"
> library). There are hundreds of bundles contained in the repository.
>
> The repository meets the following criteria:
>
> * Every jar file in the repository is a valid OSGi bundle. Any jar
> you download from the repository can be deployed as-is into an
> OSGi Service Platform and the SpringSource dm Server. It can also
> be used as a regular jar file outside of OSGi.
> * Every bundle and library has full version information associated
> with it. The package export information for a bundle contains
> version information, and the package import information for a
> bundle contains full version range compatibility information.
> * The repository is transitively complete. The mandatory
> dependencies of any bundle are guaranteed to also be in the
> repository. Most of the optional dependencies of any bundle in the
> repository will also be present. The bundles listed in any library
> definition are guaranteed to be in the repository.
> * The repository is self-consistent. Before any artefact is uploaded
> to the repository, we verify that it can be installed, resolved,
> and started in an OSGi Service Platform (using the same profile as
> the SpringSource dm Server) alongside all of the other bundles in
> the repository.
> * The repository can be used from Ivy and Maven based builds.
>
> Cheers,
>
> Peter.
>
> Niclas Hedhman wrote: > > On Fri, Oct 16, 2009 at 1:04 PM, Niclas Hedhman <
> niclas@hedhman.org> wro...
>
>
Re: Separating out RMIClassLoader dependencies
Posted by Niclas Hedhman <ni...@hedhman.org>.
A big chunk of your bundle analysis already exist in a widely used OSGi
tool, named BND. Most OSGi developers use it, since it is near impossible to
create the metadata by hand, especially the "uses" constraint.
But before that we spoke of figuring out how a particular implementation
wire up a random set of bundles. That is not predictable, nor does the OSGi
spec provides an API to find out. Well, kind of... If you have the loaded
class, you can ask from which bundle it came, but unfortunately the
StackElement doesn't give you the Class, only the class name, which isn't
enough.
But I agree with you that with Jini tech and OSGi's well-defined
classloading semantics one can do really cool stuff. Including getting rid
of some of the more troubling Jini aspects (codebase).
-- Niclas
On Oct 16, 2009 8:21 PM, "Peter Firmstone" <ji...@zeus.net.au> wrote:
Hmm, that sounds like opportunity.
A dedicated codebase service has all the time in the world to burn processor
cycles on analysis, as its main task is simply serving up jar files over
networks.
Bytecode analysis, identifies class, package and possibly module API, which
can be stored with the harvested metadata in mirror objects, one for each
class, package and module. Optional Package metadata could be a potential
source of information too.
Bundles, depend upon and export packages. Once the API is identified using
bytecode analysis, fast comparison using mirror objects could identify
compatibility.
This could be utilised in two ways:
1. As a check of backward compatibility for modules / packages over
different release versions, for substituting later compatible bundle
versions, if desired.
2. When a dependency on a package only utilises a subset of that package,
the actual API requirements may still be satisfied, even though different
release versions of that package may not usually be interchangeable or fully
compatible.
A module or bundle would exist within its own classloader, in the local jvm.
The packages or modules upon which it depends could be made available from
other classloaders.
Bundles could be uploaded to codebase services or a codebase service could
retrieve the bundles from designated repository's, perform analysis, then
make the bundles available in a location independent manner, to prevent
codebase loss and allow for redundant codebase services.
Then all one need do is to upload application bundles to the codebase server
and register a service, the service and bytecode could be provided
independently, the codebase service can provide entire application bytecode
built on Jini services and other third party libraries.
Have a look on springsource, there are many OSGi jar bundles available,
these are simply jar files with Metadata.
Someone's done so much hard work already, why not ride the wave? There are
already support tools available to create application bundle manifests.
The reason I'm considering bundles, is it reduces the number of classloaders
required, one per bundle as opposed to one per package. One cannot rely on
standard java Package meta data to exist in jar files.
********This doesn't require an OSGi framework.*********
http://blog.springsource.com/2008/02/18/creating-osgi-bundles/
http://www.springsource.com/repository/app/bundle?query=A
http://www.springsource.com/repository/app/faq
>From the website:
What is the SpringSource Enterprise Bundle Repository?
The SpringSource Enterprise Bundle Repository is a collection of open source
libraries commonly used for developing enterprise Java applications with the
Spring Framework. The repository contains jar files (bundles) and library
definition (".libd") files. A library defines a collection of bundles that
are often used together for some purpose (e.g. the "Spring Framework"
library). There are hundreds of bundles contained in the repository.
The repository meets the following criteria:
* Every jar file in the repository is a valid OSGi bundle. Any jar
you download from the repository can be deployed as-is into an
OSGi Service Platform and the SpringSource dm Server. It can also
be used as a regular jar file outside of OSGi.
* Every bundle and library has full version information associated
with it. The package export information for a bundle contains
version information, and the package import information for a
bundle contains full version range compatibility information.
* The repository is transitively complete. The mandatory
dependencies of any bundle are guaranteed to also be in the
repository. Most of the optional dependencies of any bundle in the
repository will also be present. The bundles listed in any library
definition are guaranteed to be in the repository.
* The repository is self-consistent. Before any artefact is uploaded
to the repository, we verify that it can be installed, resolved,
and started in an OSGi Service Platform (using the same profile as
the SpringSource dm Server) alongside all of the other bundles in
the repository.
* The repository can be used from Ivy and Maven based builds.
Cheers,
Peter.
Niclas Hedhman wrote: > > On Fri, Oct 16, 2009 at 1:04 PM, Niclas Hedhman <
niclas@hedhman.org> wro...
Re: Separating out RMIClassLoader dependencies
Posted by Peter Firmstone <ji...@zeus.net.au>.
Hmm, that sounds like opportunity.
A dedicated codebase service has all the time in the world to burn
processor cycles on analysis, as its main task is simply serving up jar
files over networks.
Bytecode analysis, identifies class, package and possibly module API,
which can be stored with the harvested metadata in mirror objects, one
for each class, package and module. Optional Package metadata could be
a potential source of information too.
Bundles, depend upon and export packages. Once the API is identified
using bytecode analysis, fast comparison using mirror objects could
identify compatibility.
This could be utilised in two ways:
1. As a check of backward compatibility for modules / packages over
different release versions, for substituting later compatible bundle
versions, if desired.
2. When a dependency on a package only utilises a subset of that
package, the actual API requirements may still be satisfied, even though
different release versions of that package may not usually be
interchangeable or fully compatible.
A module or bundle would exist within its own classloader, in the local
jvm. The packages or modules upon which it depends could be made
available from other classloaders.
Bundles could be uploaded to codebase services or a codebase service
could retrieve the bundles from designated repository's, perform
analysis, then make the bundles available in a location independent
manner, to prevent codebase loss and allow for redundant codebase services.
Then all one need do is to upload application bundles to the codebase
server and register a service, the service and bytecode could be
provided independently, the codebase service can provide entire
application bytecode built on Jini services and other third party libraries.
Have a look on springsource, there are many OSGi jar bundles available,
these are simply jar files with Metadata.
Someone's done so much hard work already, why not ride the wave? There
are already support tools available to create application bundle manifests.
The reason I'm considering bundles, is it reduces the number of
classloaders required, one per bundle as opposed to one per package.
One cannot rely on standard java Package meta data to exist in jar files.
********This doesn't require an OSGi framework.*********
http://blog.springsource.com/2008/02/18/creating-osgi-bundles/
http://www.springsource.com/repository/app/bundle?query=A
http://www.springsource.com/repository/app/faq
From the website:
What is the SpringSource Enterprise Bundle Repository?
The SpringSource Enterprise Bundle Repository is a collection of open
source libraries commonly used for developing enterprise Java
applications with the Spring Framework. The repository contains jar
files (bundles) and library definition (".libd") files. A library
defines a collection of bundles that are often used together for some
purpose (e.g. the "Spring Framework" library). There are hundreds of
bundles contained in the repository.
The repository meets the following criteria:
* Every jar file in the repository is a valid OSGi bundle. Any jar
you download from the repository can be deployed as-is into an
OSGi Service Platform and the SpringSource dm Server. It can also
be used as a regular jar file outside of OSGi.
* Every bundle and library has full version information associated
with it. The package export information for a bundle contains
version information, and the package import information for a
bundle contains full version range compatibility information.
* The repository is transitively complete. The mandatory
dependencies of any bundle are guaranteed to also be in the
repository. Most of the optional dependencies of any bundle in the
repository will also be present. The bundles listed in any library
definition are guaranteed to be in the repository.
* The repository is self-consistent. Before any artefact is uploaded
to the repository, we verify that it can be installed, resolved,
and started in an OSGi Service Platform (using the same profile as
the SpringSource dm Server) alongside all of the other bundles in
the repository.
* The repository can be used from Ivy and Maven based builds.
Cheers,
Peter.
Niclas Hedhman wrote:
> On Fri, Oct 16, 2009 at 1:04 PM, Niclas Hedhman <ni...@hedhman.org> wrote:
>
>> On Thu, Oct 15, 2009 at 11:53 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
>>
>>
>>> It may be possible to segregate OSGi modules, into separate ClassLoaders
>>> using bytecode dependency analysis and by capturing OSGi metadata.
>>>
>> Unfortunately, this is harder than you might think. OSGi metadata
>> provides a set of constraints, but often there are multiple solutions
>> to those constraints, and sometimes the framework is not capable of
>> resolving those constraints even if there is a solution available,
>> since exhaustive checks are too expensive for large applications. What
>> I am trying to say is; Being able to figure out how a particular
>> framework implementation has wired up the class spaces, just by
>> looking at the metadata is impossible, and likewise with bytecode
>> dependency analysis.
>>
>
> One more thing, it even depends on how the OSGi framework is started,
> since some packages may be exported from the classpath by the
> framework, and on top of that there is bootclasspath delegation
> throwing a big wrench into the heap of problems.
>
>
> Cheers
>
Re: Separating out RMIClassLoader dependencies
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Fri, Oct 16, 2009 at 1:04 PM, Niclas Hedhman <ni...@hedhman.org> wrote:
> On Thu, Oct 15, 2009 at 11:53 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
>
>> It may be possible to segregate OSGi modules, into separate ClassLoaders
>> using bytecode dependency analysis and by capturing OSGi metadata.
>
> Unfortunately, this is harder than you might think. OSGi metadata
> provides a set of constraints, but often there are multiple solutions
> to those constraints, and sometimes the framework is not capable of
> resolving those constraints even if there is a solution available,
> since exhaustive checks are too expensive for large applications. What
> I am trying to say is; Being able to figure out how a particular
> framework implementation has wired up the class spaces, just by
> looking at the metadata is impossible, and likewise with bytecode
> dependency analysis.
One more thing, it even depends on how the OSGi framework is started,
since some packages may be exported from the classpath by the
framework, and on top of that there is bootclasspath delegation
throwing a big wrench into the heap of problems.
Cheers
--
Niclas Hedhman, Software Developer
http://www.qi4j.org - New Energy for Java
I live here; http://tinyurl.com/2qq9er
I work here; http://tinyurl.com/2ymelc
I relax here; http://tinyurl.com/2cgsug
Re: Separating out RMIClassLoader dependencies
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Thu, Oct 15, 2009 at 11:53 AM, Peter Firmstone <ji...@zeus.net.au> wrote:
> It may be possible to segregate OSGi modules, into separate ClassLoaders
> using bytecode dependency analysis and by capturing OSGi metadata.
Unfortunately, this is harder than you might think. OSGi metadata
provides a set of constraints, but often there are multiple solutions
to those constraints, and sometimes the framework is not capable of
resolving those constraints even if there is a solution available,
since exhaustive checks are too expensive for large applications. What
I am trying to say is; Being able to figure out how a particular
framework implementation has wired up the class spaces, just by
looking at the metadata is impossible, and likewise with bytecode
dependency analysis.
Cheers
--
Niclas Hedhman, Software Developer
http://www.qi4j.org - New Energy for Java
I live here; http://tinyurl.com/2qq9er
I work here; http://tinyurl.com/2ymelc
I relax here; http://tinyurl.com/2cgsug
Re: Separating out RMIClassLoader dependencies
Posted by Peter Firmstone <ji...@zeus.net.au>.
Niclas Hedhman wrote:
> On Thu, Oct 15, 2009 at 12:42 AM, Gregg Wonderly <gr...@wonderly.org> wrote:
>
>
>> Anyone have thoughts or questions?
>>
>
> For OSGi (maybe Netbeans as well) there might be additional
> "problems", since "loading a class" is not a straight forward process
> of calling a static provider. In OSGi, each "module" is wired up with
> a "class space" but the same class name (difference version, different
> provider, etc) may be in another "class space" in other modules. So,
> to fulfill a contract where a static method gets called from
> "anywhere" and I need to figure out which class to load is near
> impossible (at least in a platform independent way) in OSGi, as you
> will have to figure out which bundle (module) the caller reside in, in
> an environment where multiple versions of the same class can exist.
> (The reason this is near impossible is that the OSGi spec doesn't have
> any Classloader API, from which the Bundle can be retrieved.)
>
> "Class space" is a complex and powerful concept in OSGi, which
> basically try to ensure that there are no class version clashes and
> yet provide the class visibility needed.
>
I've been pondering similar issues with code base services combined with
static bytecode dependency analysis and corresponding Classloader
hierarchies.
It may be possible to segregate OSGi modules, into separate ClassLoaders
using bytecode dependency analysis and by capturing OSGi metadata. I'm
planning on using codebase services to distribute it. Similarly,
Package version metadata could also be collected. Class Metadata, could
be distributed and stored with the bytecode as class mirror objects,
where local Classloaders could segregate class files based on received
class metadata.
The metadata (class mirror objects) could also potentially be marshalled
along with any serialized object instances to preserve class version and
dependency information of distributed objects for reassembly using
compatible classes, either identical class versions or later compatible
versions (where compatible means that an entire package or module has
maintained backward compatibility or the mirrors determine that the
dependency requirements have been satisfied).
The Classloader would then be location independent, but package or
module version dependant. The codebase might elect to distribute a
later version of a compatible package where possible when first required
locally. Class Mirrors would be used to determine compatibility.
The problem then would be to figure out how such a thing could be
shoehorned into an IDE, a problem domain that I don't currently have a
good understanding for. But for me that's another problem.
> OTOH, Paremus has shown that
>
> a) Locally available classes can be used,
> b) Remote classes be downloaded and wrapped in temporary bundles with
> OSGi contracts honored.
> c) PreferredClassLoader is not needed.
>
> So, perhaps your ambitions are too great and that you can/should
> ignore OSGi's needs.
>
>
> Cheers
>
Re: Separating out RMIClassLoader dependencies
Posted by Peter Firmstone <ji...@zeus.net.au>.
Gregg,
Just thought I'd bring this back from previous list discussion, as it
displays some of the issues you had to overcome.
Cheers,
Peter.
Gregg Wonderly wrote:
> Okay, I've done some more digging and thinking and one of the primary
> issues I believe, comes down to how "containers", whether they are
> IDEs, standards like OSGi or JEE etc, provide access to the
> "classpath" of the running environment.
>
> At the root of the behavior of PreferredClassProvider is
>
> private static ClassLoader getRMIContextClassLoader() ...
>
> which just gets the context class loader. This loader becomes the
> parent of the created PreferredClassLoader.
>
> One of the issues for me, is that I think that honoring the
> PREFERRED.LIST is an important part of bringing advertised services
> into any JVM.
>
> The primary driver of the use of PreferredClassProvider is of course
> the user of the RMIClassLoaderSpi class in the RMIClassLoader. In
> reggie, the places that deal with unmarshalling, are using
> MarshalledInstance, MarshalledWrapper and MarshalInputStream to
> package this activity.
>
> But, what I am seeing in netbeans is that reggie's unmarshalling is
> using the JVM system class loader as the parent, and that's causing
> non-preferred classes to be resolved in that class loader. However,
> my netbeans modules have a different copy of the jsk-platform.jar that
> is visible to them, and that's where they are resolving things like
> Matches and other classes that are at the core of service
> unmarshalling by reggie.
>
> So, I am trying to gather some more information to have some better
> perspectives on what actually makes the most sense.
>
> Technically, netbeans sets the context class loader to the "netbeans"
> system class loader. In most of my jini applications, I always use
>
> Thread.currentThread().setContextClassLoader(
> getClass().getClassLoader() )
>
> in newly created threads and in EDT listener objects (via the
> SyncThread class at http://swingutil.dev.java.net). I think this
> should make things work out in integrating Jini into netbeans too, but
> I've somehow not recognized the right problem yet and got it solved.
>
> Gregg Wonderly
>
>
> Niclas Hedhman wrote:
>> On Thu, Oct 15, 2009 at 12:42 AM, Gregg Wonderly <gr...@wonderly.org>
>> wrote:
>>
>>> Anyone have thoughts or questions?
>>
>> For OSGi (maybe Netbeans as well) there might be additional
>> "problems", since "loading a class" is not a straight forward process
>> of calling a static provider. In OSGi, each "module" is wired up with
>> a "class space" but the same class name (difference version, different
>> provider, etc) may be in another "class space" in other modules. So,
>> to fulfill a contract where a static method gets called from
>> "anywhere" and I need to figure out which class to load is near
>> impossible (at least in a platform independent way) in OSGi, as you
>> will have to figure out which bundle (module) the caller reside in, in
>> an environment where multiple versions of the same class can exist.
>> (The reason this is near impossible is that the OSGi spec doesn't have
>> any Classloader API, from which the Bundle can be retrieved.)
>>
>> "Class space" is a complex and powerful concept in OSGi, which
>> basically try to ensure that there are no class version clashes and
>> yet provide the class visibility needed.
>>
>> OTOH, Paremus has shown that
>>
>> a) Locally available classes can be used,
>> b) Remote classes be downloaded and wrapped in temporary bundles with
>> OSGi contracts honored.
>> c) PreferredClassLoader is not needed.
>>
>> So, perhaps your ambitions are too great and that you can/should
>> ignore OSGi's needs.
>>
>>
>> Cheers
>
>
Re: Separating out RMIClassLoader dependencies
Posted by Gregg Wonderly <ge...@cox.net>.
Okay, I've done some more digging and thinking and one of the primary issues I
believe, comes down to how "containers", whether they are IDEs, standards like
OSGi or JEE etc, provide access to the "classpath" of the running environment.
At the root of the behavior of PreferredClassProvider is
private static ClassLoader getRMIContextClassLoader() ...
which just gets the context class loader. This loader becomes the parent of the
created PreferredClassLoader.
One of the issues for me, is that I think that honoring the PREFERRED.LIST is an
important part of bringing advertised services into any JVM.
The primary driver of the use of PreferredClassProvider is of course the user of
the RMIClassLoaderSpi class in the RMIClassLoader. In reggie, the places that
deal with unmarshalling, are using MarshalledInstance, MarshalledWrapper and
MarshalInputStream to package this activity.
But, what I am seeing in netbeans is that reggie's unmarshalling is using the
JVM system class loader as the parent, and that's causing non-preferred classes
to be resolved in that class loader. However, my netbeans modules have a
different copy of the jsk-platform.jar that is visible to them, and that's where
they are resolving things like Matches and other classes that are at the core of
service unmarshalling by reggie.
So, I am trying to gather some more information to have some better perspectives
on what actually makes the most sense.
Technically, netbeans sets the context class loader to the "netbeans" system
class loader. In most of my jini applications, I always use
Thread.currentThread().setContextClassLoader( getClass().getClassLoader() )
in newly created threads and in EDT listener objects (via the SyncThread class
at http://swingutil.dev.java.net). I think this should make things work out in
integrating Jini into netbeans too, but I've somehow not recognized the right
problem yet and got it solved.
Gregg Wonderly
Niclas Hedhman wrote:
> On Thu, Oct 15, 2009 at 12:42 AM, Gregg Wonderly <gr...@wonderly.org> wrote:
>
>> Anyone have thoughts or questions?
>
> For OSGi (maybe Netbeans as well) there might be additional
> "problems", since "loading a class" is not a straight forward process
> of calling a static provider. In OSGi, each "module" is wired up with
> a "class space" but the same class name (difference version, different
> provider, etc) may be in another "class space" in other modules. So,
> to fulfill a contract where a static method gets called from
> "anywhere" and I need to figure out which class to load is near
> impossible (at least in a platform independent way) in OSGi, as you
> will have to figure out which bundle (module) the caller reside in, in
> an environment where multiple versions of the same class can exist.
> (The reason this is near impossible is that the OSGi spec doesn't have
> any Classloader API, from which the Bundle can be retrieved.)
>
> "Class space" is a complex and powerful concept in OSGi, which
> basically try to ensure that there are no class version clashes and
> yet provide the class visibility needed.
>
> OTOH, Paremus has shown that
>
> a) Locally available classes can be used,
> b) Remote classes be downloaded and wrapped in temporary bundles with
> OSGi contracts honored.
> c) PreferredClassLoader is not needed.
>
> So, perhaps your ambitions are too great and that you can/should
> ignore OSGi's needs.
>
>
> Cheers
Re: Separating out RMIClassLoader dependencies
Posted by Niclas Hedhman <ni...@hedhman.org>.
On Thu, Oct 15, 2009 at 12:42 AM, Gregg Wonderly <gr...@wonderly.org> wrote:
> Anyone have thoughts or questions?
For OSGi (maybe Netbeans as well) there might be additional
"problems", since "loading a class" is not a straight forward process
of calling a static provider. In OSGi, each "module" is wired up with
a "class space" but the same class name (difference version, different
provider, etc) may be in another "class space" in other modules. So,
to fulfill a contract where a static method gets called from
"anywhere" and I need to figure out which class to load is near
impossible (at least in a platform independent way) in OSGi, as you
will have to figure out which bundle (module) the caller reside in, in
an environment where multiple versions of the same class can exist.
(The reason this is near impossible is that the OSGi spec doesn't have
any Classloader API, from which the Bundle can be retrieved.)
"Class space" is a complex and powerful concept in OSGi, which
basically try to ensure that there are no class version clashes and
yet provide the class visibility needed.
OTOH, Paremus has shown that
a) Locally available classes can be used,
b) Remote classes be downloaded and wrapped in temporary bundles with
OSGi contracts honored.
c) PreferredClassLoader is not needed.
So, perhaps your ambitions are too great and that you can/should
ignore OSGi's needs.
Cheers
--
Niclas Hedhman, Software Developer
http://www.qi4j.org - New Energy for Java
I live here; http://tinyurl.com/2qq9er
I work here; http://tinyurl.com/2ymelc
I relax here; http://tinyurl.com/2cgsug