You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@river.apache.org by Christopher Dolan <ch...@avid.com> on 2010/08/20 17:10:07 UTC
RE: [jira] Updated: (RIVER-336) Jini should support platforms other than those with RMIClassLoader as the classloading control point. IDEs inparticular need help.
I hereby extend a public THANK YOU to Gregg for this excellent patch,
and to recommend that it be accepted into River. With this patch
(actually, a slightly trimmed down version of it), I am successfully
running a River service from inside an OSGi container (Equinox) for the
first time with my custom RMIClassLoaderSpi implementation.
Chris
-----Original Message-----
From: Gregg Wonderly [mailto:gregg@wonderly.org]
Sent: Friday, April 02, 2010 12:31 AM
To: river-dev@incubator.apache.org
Subject: Re: [jira] Updated: (RIVER-336) Jini should support platforms
other than those with RMIClassLoader as the classloading control point.
IDEs inparticular need help.
There are two primary issues that I needed to deal with for netbeans.
First, because netbeans is an application with a native launcher, and
all kinds
of things about it, as a platform, are private, I had problems getting
the
RMIClassLoaderSPI in River, PerferredClassProvider, to even activate.
There was
an issue of having to copy the jars into a private directory. That
wasn't
something that felt like a production step, and it would not make it
possible to
install and use a plugin supporting river without restarting the IDE.
This
seemed a bit fragile if not just ugly.
Second, the simple implementation of PreferredClassLoader creation in
net.jini.loader just uses the system classloader as the "parent" for the
created
class loader. In netbeans, the class loader hierachy that a module
sees, is
just the module and its dependents, with the parent of that loader being
the
system class loader. The netbeans system class loader is nearly empty.
It just
has the bootstrap of netbeans visible, and then netbeans creates
compartmentalized views of various modules as it loads things. So, when
I
finally got PreferredClassProvider activated, downloaded jars could not
find
stuff that was "platform" in the since that the module had this stuff in
its
classloader. That module classloader was not visible because the
PreferredClassLoader's parent was just the netbeans startup jars and the
JRE.
Thus, I really had a hard time making net.jini.loader work as it
existed. I
looked around and saw that there are actually very few places that
RMIClassLoaderSPI et.al. is used, and that all of those places where of
interest
to me in how I could make Jini work inside of netbeans (at least I
hoped).
So, I decided to just put a layer of indirection (what programmers do to
solve
most problems :-) between net.jini.loader and RMIClassLoaderSPI that
would allow
the mechanism itself to be supplanted.
So, these patches allow one to override how all of the RMIClassLoaderSPI
mechanisms are implemented, plus adding some twists specifically about
how the
"parent" classloader is chosen for PreferredClassLoader instances.
And, to top it off, some of you may recall that I modified preferred
class
loading to actually have the facility to "never prefer" some classes.
What this
meant was that jar files in the codebase would never be downloaded and
thus in
large, distributed systems, with lots of jars visible across slow
networks, a
ServiceUI client application could startup, and show all visible
services,
without downloading anything.
Additionally, I also make changes to Reggie to return marshalled data
values so
that unmarshalling would not occur, which would trigger downloads.
By making these changes, I was able to move the "never prefer"
implementation
into my CodebaseClassAccess implementation so that it becomes a feature
for me,
not a platform implementation detail.
Please ask questions if I've not provided enough details somewhere.
Gregg Wonderly
Christopher Dolan wrote:
> Gregg,
>
> This patch is interesting to me.
>
> The implementation is clear: create a new plugin point to override the
> default RMIClassLoader. Could you explain the motivation a little
more?
> It seems like the intention is to make River play nice in a large
> process by not insisting that the global RMIClassLoader be configured
to
> suit River. And it looks like it is fully backward compatible for
River
> deployments that are already overriding RMIClassLoader by defaulting
to
> RMIClassLoaderCodebaseAccess.
>
> If nothing else, I would love to use something like this just to
reduce
> the huge number of places where I have to include
> "-Djava.rmi.server.RMIClassLoaderSpi=...". That alone would be a win.
>
> Or am I reading too much into this?
>
> Chris
>
> -----Original Message-----
> From: Peter Firmstone [mailto:jini@zeus.net.au]
> Sent: Thursday, April 01, 2010 4:16 AM
> To: river-dev@incubator.apache.org
> Subject: Re: [jira] Updated: (RIVER-336) Jini should support platforms
> other than those with RMIClassLoader as the classloading control
point.
> IDEs inparticular need help.
>
> Should I create a branch on svn for this?
>
> Regards,
>
> Peter.
>
> Gregg Wonderly (JIRA) wrote:
>> [
>
https://issues.apache.org/jira/browse/RIVER-336?page=com.atlassian.jira.
> plugin.system.issuetabpanels:all-tabpanel ]
>> Gregg Wonderly updated RIVER-336:
>> ---------------------------------
>>
>> Attachment: rmicl.diff.txt
>>
>> This is a diff out of my perforce node for the affected classes that
> I've been using for some time. The changes shown here are
preliminary
> and should be considered experimental.
>>
>>> Jini should support platforms other than those with RMIClassLoader
as
> the classloading control point. IDEs inparticular need help.
>
------------------------------------------------------------------------
> -----------------------------------------------------------
>>> Key: RIVER-336
>>> URL: https://issues.apache.org/jira/browse/RIVER-336
>>> Project: River
>>> Issue Type: New Feature
>>> Components: net_jini_loader
>>> Affects Versions: AR3
>>> Reporter: Gregg Wonderly
>>> Attachments: rmicl.diff.txt
>>>
>>>
>>> The RMIClassLoader class and RMIClassLoaderSPI is currently the
> control point for managing the "platform" view of how classes are
> loaded. In IDEs and other different environments, the "parent"
> classloader view, is not always the "system class loader". There are
> some other variations on class loading that seem to indicate that
while
> RMIClassLoaderSPI can be plugged into, it doesn't always provide quite
> the right facilities because even plugging into the system class
loader
> to override it might not be possible.
>>> The diffs included here show some preliminary work that I did
> investigating this issue to try and make it possible to discover and
> load Jini servers within the netbeans IDE.
>>> Refinement and some rework will be needed, and some other
> investigation into other platforms such as JEE and other IDEs would be
> helpful in making sure we understand what is really needed. Even OSGi
> would be something to look at.
>>>
>>
>
>
RE: [jira] Updated: (RIVER-336) Jini should support platforms other than those with RMIClassLoader as the classloading control point. IDEs inparticular need help.
Posted by Christopher Dolan <ch...@avid.com>.
For others going down this same road, here's a bit of acquired wisdom:
it's important that you do NOT set the system property
java.rmi.server.RMIClassLoaderSpi because if you do then you'll get
NoClassDefFoundError when com.sun.jini.lookup.entry.LookupAttributes
calls the java.rmi.MarshalledObject constructor and
java.rmi.server.RMIClassLoader crashes on class initialization trying to
find your custom SPI.
In my code, I do this during bootstrap:
String myLoaderSPI =
System.getProperty("java.rmi.server.RMIClassLoaderSpi");
if (null != myLoaderSPI && null ==
ClassLoader.getSystemResource(myLoaderSPI)) {
// HACK! We can't force our SPI into the system class
loader, so we
// need to back it out because otherwise it breaks
MarshalledObject
// usage in the com.sun.jini.lookup.entry.LookupAttributes
class.
System.setProperty("java.rmi.server.RMIClassLoaderSpi",
"default");
}
Another big gotcha is that you may not be able to talk to older Reggie
instances because of the hardcoded RMIClassLoader reference in
ServiceTypeBase in reggie-dl.jar. I worked around this via a custom
URLStreamHandlerFactory that detects an out-of-date reggie-dl.jar and
substitutes a newer version invisibly at runtime. This is a very
unpleasant hack, but I could find no alternative. I suppose this could
also be solved via a preferred class...
Chris
-----Original Message-----
From: Christopher Dolan [mailto:christopher.dolan@avid.com]
Sent: Friday, August 20, 2010 10:10 AM
To: river-dev@incubator.apache.org
Subject: RE: [jira] Updated: (RIVER-336) Jini should support platforms
other than those with RMIClassLoader as the classloading control point.
IDEs inparticular need help.
I hereby extend a public THANK YOU to Gregg for this excellent patch,
and to recommend that it be accepted into River. With this patch
(actually, a slightly trimmed down version of it), I am successfully
running a River service from inside an OSGi container (Equinox) for the
first time with my custom RMIClassLoaderSpi implementation.
Chris
-----Original Message-----
From: Gregg Wonderly [mailto:gregg@wonderly.org]
Sent: Friday, April 02, 2010 12:31 AM
To: river-dev@incubator.apache.org
Subject: Re: [jira] Updated: (RIVER-336) Jini should support platforms
other than those with RMIClassLoader as the classloading control point.
IDEs inparticular need help.
There are two primary issues that I needed to deal with for netbeans.
First, because netbeans is an application with a native launcher, and
all kinds
of things about it, as a platform, are private, I had problems getting
the
RMIClassLoaderSPI in River, PerferredClassProvider, to even activate.
There was
an issue of having to copy the jars into a private directory. That
wasn't
something that felt like a production step, and it would not make it
possible to
install and use a plugin supporting river without restarting the IDE.
This
seemed a bit fragile if not just ugly.
Second, the simple implementation of PreferredClassLoader creation in
net.jini.loader just uses the system classloader as the "parent" for the
created
class loader. In netbeans, the class loader hierachy that a module
sees, is
just the module and its dependents, with the parent of that loader being
the
system class loader. The netbeans system class loader is nearly empty.
It just
has the bootstrap of netbeans visible, and then netbeans creates
compartmentalized views of various modules as it loads things. So, when
I
finally got PreferredClassProvider activated, downloaded jars could not
find
stuff that was "platform" in the since that the module had this stuff in
its
classloader. That module classloader was not visible because the
PreferredClassLoader's parent was just the netbeans startup jars and the
JRE.
Thus, I really had a hard time making net.jini.loader work as it
existed. I
looked around and saw that there are actually very few places that
RMIClassLoaderSPI et.al. is used, and that all of those places where of
interest
to me in how I could make Jini work inside of netbeans (at least I
hoped).
So, I decided to just put a layer of indirection (what programmers do to
solve
most problems :-) between net.jini.loader and RMIClassLoaderSPI that
would allow
the mechanism itself to be supplanted.
So, these patches allow one to override how all of the RMIClassLoaderSPI
mechanisms are implemented, plus adding some twists specifically about
how the
"parent" classloader is chosen for PreferredClassLoader instances.
And, to top it off, some of you may recall that I modified preferred
class
loading to actually have the facility to "never prefer" some classes.
What this
meant was that jar files in the codebase would never be downloaded and
thus in
large, distributed systems, with lots of jars visible across slow
networks, a
ServiceUI client application could startup, and show all visible
services,
without downloading anything.
Additionally, I also make changes to Reggie to return marshalled data
values so
that unmarshalling would not occur, which would trigger downloads.
By making these changes, I was able to move the "never prefer"
implementation
into my CodebaseClassAccess implementation so that it becomes a feature
for me,
not a platform implementation detail.
Please ask questions if I've not provided enough details somewhere.
Gregg Wonderly
Christopher Dolan wrote:
> Gregg,
>
> This patch is interesting to me.
>
> The implementation is clear: create a new plugin point to override the
> default RMIClassLoader. Could you explain the motivation a little
more?
> It seems like the intention is to make River play nice in a large
> process by not insisting that the global RMIClassLoader be configured
to
> suit River. And it looks like it is fully backward compatible for
River
> deployments that are already overriding RMIClassLoader by defaulting
to
> RMIClassLoaderCodebaseAccess.
>
> If nothing else, I would love to use something like this just to
reduce
> the huge number of places where I have to include
> "-Djava.rmi.server.RMIClassLoaderSpi=...". That alone would be a win.
>
> Or am I reading too much into this?
>
> Chris
>
> -----Original Message-----
> From: Peter Firmstone [mailto:jini@zeus.net.au]
> Sent: Thursday, April 01, 2010 4:16 AM
> To: river-dev@incubator.apache.org
> Subject: Re: [jira] Updated: (RIVER-336) Jini should support platforms
> other than those with RMIClassLoader as the classloading control
point.
> IDEs inparticular need help.
>
> Should I create a branch on svn for this?
>
> Regards,
>
> Peter.
>
> Gregg Wonderly (JIRA) wrote:
>> [
>
https://issues.apache.org/jira/browse/RIVER-336?page=com.atlassian.jira.
> plugin.system.issuetabpanels:all-tabpanel ]
>> Gregg Wonderly updated RIVER-336:
>> ---------------------------------
>>
>> Attachment: rmicl.diff.txt
>>
>> This is a diff out of my perforce node for the affected classes that
> I've been using for some time. The changes shown here are
preliminary
> and should be considered experimental.
>>
>>> Jini should support platforms other than those with RMIClassLoader
as
> the classloading control point. IDEs inparticular need help.
>
------------------------------------------------------------------------
> -----------------------------------------------------------
>>> Key: RIVER-336
>>> URL: https://issues.apache.org/jira/browse/RIVER-336
>>> Project: River
>>> Issue Type: New Feature
>>> Components: net_jini_loader
>>> Affects Versions: AR3
>>> Reporter: Gregg Wonderly
>>> Attachments: rmicl.diff.txt
>>>
>>>
>>> The RMIClassLoader class and RMIClassLoaderSPI is currently the
> control point for managing the "platform" view of how classes are
> loaded. In IDEs and other different environments, the "parent"
> classloader view, is not always the "system class loader". There are
> some other variations on class loading that seem to indicate that
while
> RMIClassLoaderSPI can be plugged into, it doesn't always provide quite
> the right facilities because even plugging into the system class
loader
> to override it might not be possible.
>>> The diffs included here show some preliminary work that I did
> investigating this issue to try and make it possible to discover and
> load Jini servers within the netbeans IDE.
>>> Refinement and some rework will be needed, and some other
> investigation into other platforms such as JEE and other IDEs would be
> helpful in making sure we understand what is really needed. Even OSGi
> would be something to look at.
>>>
>>
>
>
Re: [jira] Updated: (RIVER-336) Jini should support platforms other
than those with RMIClassLoader as the classloading control point. IDEs inparticular
need help.
Posted by Peter Firmstone <ji...@zeus.net.au>.
It's currently included in svn, for review, also a slightly trimmed
version of it.
Could you share the goodies on some of your implementation details?
Gregg's been very helpful, assisting to guide us in the right direction,
what problems remain to be solved, delayed proxy unmarshalling is
another one.
Peter.
Christopher Dolan wrote:
> I hereby extend a public THANK YOU to Gregg for this excellent patch,
> and to recommend that it be accepted into River. With this patch
> (actually, a slightly trimmed down version of it), I am successfully
> running a River service from inside an OSGi container (Equinox) for the
> first time with my custom RMIClassLoaderSpi implementation.
>
> Chris
>
> -----Original Message-----
> From: Gregg Wonderly [mailto:gregg@wonderly.org]
> Sent: Friday, April 02, 2010 12:31 AM
> To: river-dev@incubator.apache.org
> Subject: Re: [jira] Updated: (RIVER-336) Jini should support platforms
> other than those with RMIClassLoader as the classloading control point.
> IDEs inparticular need help.
>
> There are two primary issues that I needed to deal with for netbeans.
>
> First, because netbeans is an application with a native launcher, and
> all kinds
> of things about it, as a platform, are private, I had problems getting
> the
> RMIClassLoaderSPI in River, PerferredClassProvider, to even activate.
> There was
> an issue of having to copy the jars into a private directory. That
> wasn't
> something that felt like a production step, and it would not make it
> possible to
> install and use a plugin supporting river without restarting the IDE.
> This
> seemed a bit fragile if not just ugly.
>
> Second, the simple implementation of PreferredClassLoader creation in
> net.jini.loader just uses the system classloader as the "parent" for the
> created
> class loader. In netbeans, the class loader hierachy that a module
> sees, is
> just the module and its dependents, with the parent of that loader being
> the
> system class loader. The netbeans system class loader is nearly empty.
> It just
> has the bootstrap of netbeans visible, and then netbeans creates
> compartmentalized views of various modules as it loads things. So, when
> I
> finally got PreferredClassProvider activated, downloaded jars could not
> find
> stuff that was "platform" in the since that the module had this stuff in
> its
> classloader. That module classloader was not visible because the
> PreferredClassLoader's parent was just the netbeans startup jars and the
> JRE.
>
> Thus, I really had a hard time making net.jini.loader work as it
> existed. I
> looked around and saw that there are actually very few places that
> RMIClassLoaderSPI et.al. is used, and that all of those places where of
> interest
> to me in how I could make Jini work inside of netbeans (at least I
> hoped).
>
> So, I decided to just put a layer of indirection (what programmers do to
> solve
> most problems :-) between net.jini.loader and RMIClassLoaderSPI that
> would allow
> the mechanism itself to be supplanted.
>
> So, these patches allow one to override how all of the RMIClassLoaderSPI
>
> mechanisms are implemented, plus adding some twists specifically about
> how the
> "parent" classloader is chosen for PreferredClassLoader instances.
>
> And, to top it off, some of you may recall that I modified preferred
> class
> loading to actually have the facility to "never prefer" some classes.
> What this
> meant was that jar files in the codebase would never be downloaded and
> thus in
> large, distributed systems, with lots of jars visible across slow
> networks, a
> ServiceUI client application could startup, and show all visible
> services,
> without downloading anything.
>
> Additionally, I also make changes to Reggie to return marshalled data
> values so
> that unmarshalling would not occur, which would trigger downloads.
>
> By making these changes, I was able to move the "never prefer"
> implementation
> into my CodebaseClassAccess implementation so that it becomes a feature
> for me,
> not a platform implementation detail.
>
> Please ask questions if I've not provided enough details somewhere.
>
> Gregg Wonderly
>
> Christopher Dolan wrote:
>
>> Gregg,
>>
>> This patch is interesting to me.
>>
>> The implementation is clear: create a new plugin point to override the
>> default RMIClassLoader. Could you explain the motivation a little
>>
> more?
>
>> It seems like the intention is to make River play nice in a large
>> process by not insisting that the global RMIClassLoader be configured
>>
> to
>
>> suit River. And it looks like it is fully backward compatible for
>>
> River
>
>> deployments that are already overriding RMIClassLoader by defaulting
>>
> to
>
>> RMIClassLoaderCodebaseAccess.
>>
>> If nothing else, I would love to use something like this just to
>>
> reduce
>
>> the huge number of places where I have to include
>> "-Djava.rmi.server.RMIClassLoaderSpi=...". That alone would be a win.
>>
>> Or am I reading too much into this?
>>
>> Chris
>>
>> -----Original Message-----
>> From: Peter Firmstone [mailto:jini@zeus.net.au]
>> Sent: Thursday, April 01, 2010 4:16 AM
>> To: river-dev@incubator.apache.org
>> Subject: Re: [jira] Updated: (RIVER-336) Jini should support platforms
>> other than those with RMIClassLoader as the classloading control
>>
> point.
>
>> IDEs inparticular need help.
>>
>> Should I create a branch on svn for this?
>>
>> Regards,
>>
>> Peter.
>>
>> Gregg Wonderly (JIRA) wrote:
>>
>>> [
>>>
> https://issues.apache.org/jira/browse/RIVER-336?page=com.atlassian.jira.
>
>> plugin.system.issuetabpanels:all-tabpanel ]
>>
>>> Gregg Wonderly updated RIVER-336:
>>> ---------------------------------
>>>
>>> Attachment: rmicl.diff.txt
>>>
>>> This is a diff out of my perforce node for the affected classes that
>>>
>> I've been using for some time. The changes shown here are
>>
> preliminary
>
>> and should be considered experimental.
>>
>>>
>>>
>>>> Jini should support platforms other than those with RMIClassLoader
>>>>
> as
>
>> the classloading control point. IDEs inparticular need help.
>>
>>
> ------------------------------------------------------------------------
>
>> -----------------------------------------------------------
>>
>>>> Key: RIVER-336
>>>> URL: https://issues.apache.org/jira/browse/RIVER-336
>>>> Project: River
>>>> Issue Type: New Feature
>>>> Components: net_jini_loader
>>>> Affects Versions: AR3
>>>> Reporter: Gregg Wonderly
>>>> Attachments: rmicl.diff.txt
>>>>
>>>>
>>>> The RMIClassLoader class and RMIClassLoaderSPI is currently the
>>>>
>> control point for managing the "platform" view of how classes are
>> loaded. In IDEs and other different environments, the "parent"
>> classloader view, is not always the "system class loader". There are
>> some other variations on class loading that seem to indicate that
>>
> while
>
>> RMIClassLoaderSPI can be plugged into, it doesn't always provide quite
>> the right facilities because even plugging into the system class
>>
> loader
>
>> to override it might not be possible.
>>
>>>> The diffs included here show some preliminary work that I did
>>>>
>> investigating this issue to try and make it possible to discover and
>> load Jini servers within the netbeans IDE.
>>
>>>> Refinement and some rework will be needed, and some other
>>>>
>> investigation into other platforms such as JEE and other IDEs would be
>> helpful in making sure we understand what is really needed. Even OSGi
>> would be something to look at.
>>
>>>>
>>>>
>>>
>>>
>>
>
>
>