You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by "Geir Magnusson Jr." <ge...@pobox.com> on 2007/02/07 05:18:54 UTC
[drlvm] The Return of the Hidden Classloader, Part II
This is an oldie but goodie..
We never solved the problem of how to hide non java*. packages that
are on the boot classpath from apps. We talked about a few
possibilities :
1) repackage them to a different package namespace
2) have a "hidden" classloader that only the system classloader can use.
I like #2. Has anyone looked into this or made any progress? Do
people think this is as important as I do?
geir
Re: [drlvm] The Return of the Hidden Classloader, Part II
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
correction below...
On Feb 7, 2007, at 9:06 AM, Geir Magnusson Jr. wrote:
>
> On Feb 7, 2007, at 7:56 AM, Tim Ellison wrote:
>
>> Geir Magnusson Jr. wrote:
>>> On Feb 7, 2007, at 7:00 AM, Tim Ellison wrote:
>>>> Rather than a hidden class loader we can filter the app
>>>> classloader's
>>>> delegation to the bootclassloader so it only gets an opportunity
>>>> to load
>>>> API types. Let me have a quick play ...
>>>
>>> Yes - that's the same concept, but I can see how that's going to be
>>> easier, because there's less machinery needed to figure out who is
>>> asking. If you don't mind, I'll take a wack.
>>
>> The hack I was thinking of is to override loadClass in
>> URLClassLoader,
>> and put a filter in there to only delegate to the parent if you
>> are the
>> system (i.e. what I would call the 'application') class loader and
>> the
>> package is one that you want to pick up.
>
> Perfect. I was looking at stuffing it into the SubURLCLassloader
> loadClass, but this is much cleaner.
>
>>
>> My first pass through only allows JSE API types, but that likely
>> won't
>> be great as it doesn't account for people putting JUnit etc. on the
>> bootclasspath and expecting it to work out. We may need to flip it
>> around and have a list of explicitly hidden packages:
>
> There are two ways I can think of :
>
> A) Have a list of system-loader verboten (mx4j, xalan, etc...)
>
> or
>
> B) Two Phase :
>
> 1) list of things on the public surface of the Java SE API :
>
> java.
> javax.
> etc
>
> so we can fail-fast for the majority of stuff and then
>
> 2) A list of things that the VM knows is allowed because they were
> placed on the bootclasspath or extension classpath by the user.
>
> and if you don't match, it's forbidden.
>
>
> I actually think that the first way is faster, since the list is
> shorter (at least right now)
>
>
>
> So the next question is for the non-static list (i.e. for option A
> or the second phase of B).
>
> This has to come from the VM, as the VM knows, and it's going to
> get messy if you want to do sophisticated things w/ the
> classloaders and such. (aka "OSGi")
>
> So...
>
> I'd like to add a new luni-kernel abstract class :
>
> o.a.h.kernel.vm.VMUtillityBase.java
>
> that has (depending on if we choose A or B) one of
>
> String[] getAppAllowedSystemLoaderList() //or something
> String[] getAppDeniedSystemLoaderList() //or something
>
> that for the Allowed version has an empty list (maybe we put JUnit
> in there...) and for the Denied version has mx4j, xalan
>
> and a class
>
> o.a.h.kernel.vm.VMUtility.java
that extends VMUtilityBase
>
> so and use VMUtility in URLClassloader to get the list. This way,
> there are no VM changes allowed now, VMs can then provide their own
"there are no VM changes *required* now"
> o.a.h.k.v.VMUtillity to reflect user-added stuff on bootclasspath
> and extentionclasspath, and in the future, we can add default
> functionality to VMUtilityBase and use it immediately w/o requiring
> changes to the VMs.
>
> Does this make any sense?
>
> geir
>
>>
>>
>> protected synchronized Class<?> loadClass(String className,
>> boolean resolveClass) throws ClassNotFoundException {
>>
>> if (this == getSystemClassLoader()) {
>> int index = className.lastIndexOf('.');
>> String pkgName = index > 0 ? className.substring(0,
>> index) : "";
>>
>> System.out.println("Checking visibility for " + pkgName);
>> if (!pkgName.startsWith("java.") && !pkgName.startsWith
>> ("javax.")
>> && !pkgName.startsWith("org.ietf.jgss")
>> && !pkgName.startsWith("org.omg")
>> && !pkgName.startsWith("org.w3c.dom")
>> && !pkgName.startsWith("org.xml.sax")) {
>> throw new ClassNotFoundException(className);
>> }
>> }
>> return super.loadClass(className, resolveClass);
>> }
>>
>
Re: [drlvm] The Return of the Hidden Classloader, Part II
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
On Feb 7, 2007, at 7:56 AM, Tim Ellison wrote:
> Geir Magnusson Jr. wrote:
>> On Feb 7, 2007, at 7:00 AM, Tim Ellison wrote:
>>> Rather than a hidden class loader we can filter the app
>>> classloader's
>>> delegation to the bootclassloader so it only gets an opportunity
>>> to load
>>> API types. Let me have a quick play ...
>>
>> Yes - that's the same concept, but I can see how that's going to be
>> easier, because there's less machinery needed to figure out who is
>> asking. If you don't mind, I'll take a wack.
>
> The hack I was thinking of is to override loadClass in URLClassLoader,
> and put a filter in there to only delegate to the parent if you are
> the
> system (i.e. what I would call the 'application') class loader and the
> package is one that you want to pick up.
Perfect. I was looking at stuffing it into the SubURLCLassloader
loadClass, but this is much cleaner.
>
> My first pass through only allows JSE API types, but that likely won't
> be great as it doesn't account for people putting JUnit etc. on the
> bootclasspath and expecting it to work out. We may need to flip it
> around and have a list of explicitly hidden packages:
There are two ways I can think of :
A) Have a list of system-loader verboten (mx4j, xalan, etc...)
or
B) Two Phase :
1) list of things on the public surface of the Java SE API :
java.
javax.
etc
so we can fail-fast for the majority of stuff and then
2) A list of things that the VM knows is allowed because they were
placed on the bootclasspath or extension classpath by the user.
and if you don't match, it's forbidden.
I actually think that the first way is faster, since the list is
shorter (at least right now)
So the next question is for the non-static list (i.e. for option A or
the second phase of B).
This has to come from the VM, as the VM knows, and it's going to get
messy if you want to do sophisticated things w/ the classloaders and
such. (aka "OSGi")
So...
I'd like to add a new luni-kernel abstract class :
o.a.h.kernel.vm.VMUtillityBase.java
that has (depending on if we choose A or B) one of
String[] getAppAllowedSystemLoaderList() //or something
String[] getAppDeniedSystemLoaderList() //or something
that for the Allowed version has an empty list (maybe we put JUnit in
there...) and for the Denied version has mx4j, xalan
and a class
o.a.h.kernel.vm.VMUtility.java
so and use VMUtility in URLClassloader to get the list. This way,
there are no VM changes allowed now, VMs can then provide their own
o.a.h.k.v.VMUtillity to reflect user-added stuff on bootclasspath and
extentionclasspath, and in the future, we can add default
functionality to VMUtilityBase and use it immediately w/o requiring
changes to the VMs.
Does this make any sense?
geir
>
>
> protected synchronized Class<?> loadClass(String className,
> boolean resolveClass) throws ClassNotFoundException {
>
> if (this == getSystemClassLoader()) {
> int index = className.lastIndexOf('.');
> String pkgName = index > 0 ? className.substring(0,
> index) : "";
>
> System.out.println("Checking visibility for " + pkgName);
> if (!pkgName.startsWith("java.") && !pkgName.startsWith
> ("javax.")
> && !pkgName.startsWith("org.ietf.jgss")
> && !pkgName.startsWith("org.omg")
> && !pkgName.startsWith("org.w3c.dom")
> && !pkgName.startsWith("org.xml.sax")) {
> throw new ClassNotFoundException(className);
> }
> }
> return super.loadClass(className, resolveClass);
> }
>
Re: [drlvm] The Return of the Hidden Classloader, Part II
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
On Feb 7, 2007, at 7:56 AM, Tim Ellison wrote:
>
>
> protected synchronized Class<?> loadClass(String className,
> boolean resolveClass) throws ClassNotFoundException {
>
> if (this == getSystemClassLoader()) {
> int index = className.lastIndexOf('.');
> String pkgName = index > 0 ? className.substring(0,
> index) : "";
>
> System.out.println("Checking visibility for " + pkgName);
> if (!pkgName.startsWith("java.") && !pkgName.startsWith
> ("javax.")
> && !pkgName.startsWith("org.ietf.jgss")
> && !pkgName.startsWith("org.omg")
> && !pkgName.startsWith("org.w3c.dom")
> && !pkgName.startsWith("org.xml.sax")) {
> throw new ClassNotFoundException(className);
> }
> }
> return super.loadClass(className, resolveClass);
> }
>
Revealing my ignorance about java classloading :
Why wouldn't we need to care about the caller? (This is what had me
going into the VM first...) What if loadClass() is getting called by
an implementation class, which does need access to the internal
packages that are prohibited for app classes?
geir
Re: [drlvm] The Return of the Hidden Classloader, Part II
Posted by Tim Ellison <t....@gmail.com>.
Geir Magnusson Jr. wrote:
> On Feb 7, 2007, at 7:00 AM, Tim Ellison wrote:
>> Rather than a hidden class loader we can filter the app classloader's
>> delegation to the bootclassloader so it only gets an opportunity to load
>> API types. Let me have a quick play ...
>
> Yes - that's the same concept, but I can see how that's going to be
> easier, because there's less machinery needed to figure out who is
> asking. If you don't mind, I'll take a wack.
The hack I was thinking of is to override loadClass in URLClassLoader,
and put a filter in there to only delegate to the parent if you are the
system (i.e. what I would call the 'application') class loader and the
package is one that you want to pick up.
My first pass through only allows JSE API types, but that likely won't
be great as it doesn't account for people putting JUnit etc. on the
bootclasspath and expecting it to work out. We may need to flip it
around and have a list of explicitly hidden packages:
protected synchronized Class<?> loadClass(String className,
boolean resolveClass) throws ClassNotFoundException {
if (this == getSystemClassLoader()) {
int index = className.lastIndexOf('.');
String pkgName = index > 0 ? className.substring(0, index) : "";
System.out.println("Checking visibility for " + pkgName);
if (!pkgName.startsWith("java.") && !pkgName.startsWith("javax.")
&& !pkgName.startsWith("org.ietf.jgss")
&& !pkgName.startsWith("org.omg")
&& !pkgName.startsWith("org.w3c.dom")
&& !pkgName.startsWith("org.xml.sax")) {
throw new ClassNotFoundException(className);
}
}
return super.loadClass(className, resolveClass);
}
Re: [drlvm] The Return of the Hidden Classloader, Part II
Posted by "Geir Magnusson Jr." <ge...@pobox.com>.
On Feb 7, 2007, at 7:00 AM, Tim Ellison wrote:
> Yang Paulex wrote:
>> 2007/2/7, Geir Magnusson Jr. <ge...@pobox.com>:
>>>
>>> This is an oldie but goodie..
>>>
>>> We never solved the problem of how to hide non java*. packages that
>>> are on the boot classpath from apps. We talked about a few
>>> possibilities :
>>>
>>> 1) repackage them to a different package namespace
>>>
>>> 2) have a "hidden" classloader that only the system classloader
>>> can use.
>>>
>>> I like #2. Has anyone looked into this or made any progress? Do
>>> people think this is as important as I do?
>>
>>
>> +1, the hidden classloader so far seems a better way to go, but
>> for long
>> term, I wish the JSR 277[1] or JSR 291[2] can be considered to
>> address this
>> kind of issue in general.
>>
>> [1] http://jcp.org/en/jsr/detail?id=277
>> [2] http://jcp.org/en/jsr/detail?id=291
>
> True, but in the meantime it sounds like we need a tactical
> solution to
> the problems that Robin and Geir and others have been having.
>
> Rather than a hidden class loader we can filter the app classloader's
> delegation to the bootclassloader so it only gets an opportunity to
> load
> API types. Let me have a quick play ...
>
Yes - that's the same concept, but I can see how that's going to be
easier, because there's less machinery needed to figure out who is
asking. If you don't mind, I'll take a wack.
geir
> Regards,
> Tim
Re: [drlvm] The Return of the Hidden Classloader, Part II
Posted by Tim Ellison <t....@gmail.com>.
Yang Paulex wrote:
> 2007/2/7, Geir Magnusson Jr. <ge...@pobox.com>:
>>
>> This is an oldie but goodie..
>>
>> We never solved the problem of how to hide non java*. packages that
>> are on the boot classpath from apps. We talked about a few
>> possibilities :
>>
>> 1) repackage them to a different package namespace
>>
>> 2) have a "hidden" classloader that only the system classloader can use.
>>
>> I like #2. Has anyone looked into this or made any progress? Do
>> people think this is as important as I do?
>
>
> +1, the hidden classloader so far seems a better way to go, but for long
> term, I wish the JSR 277[1] or JSR 291[2] can be considered to address this
> kind of issue in general.
>
> [1] http://jcp.org/en/jsr/detail?id=277
> [2] http://jcp.org/en/jsr/detail?id=291
True, but in the meantime it sounds like we need a tactical solution to
the problems that Robin and Geir and others have been having.
Rather than a hidden class loader we can filter the app classloader's
delegation to the bootclassloader so it only gets an opportunity to load
API types. Let me have a quick play ...
Regards,
Tim
Re: [drlvm] The Return of the Hidden Classloader, Part II
Posted by Alex Blewitt <al...@gmail.com>.
On 07/02/07, Tim Ellison <t....@gmail.com> wrote:
>
> There are a number of reasons why using Equinox directly will not work
> for Harmony, where we are modularizing the class library itself. Not
> least of which is that the very Java types used to implement Equinox
> need to be part of the managed bundles. It's an area of ongoing work.
Of course, I didn't mean to suggest that it be based on Equinox. I was
merely pointing out that the system is configurable at runtime to
define the behaviour of what this hidden classloader does, and that it
might be useful to do something similar here.
Alex.
Re: [drlvm] The Return of the Hidden Classloader, Part II
Posted by Tim Ellison <t....@gmail.com>.
Alex Blewitt wrote:
> On 07/02/07, Yang Paulex <pa...@gmail.com> wrote:
>> 2007/2/7, Geir Magnusson Jr. <ge...@pobox.com>:
>> >
>> > We never solved the problem of how to hide non java*. packages that
>> > are on the boot classpath from apps. We talked about a few
>> > possibilities :
>> >
>> > 2) have a "hidden" classloader that only the system classloader can
>> use.
>> >
>> > I like #2. Has anyone looked into this or made any progress? Do
>> > people think this is as important as I do?
>>
>> +1, the hidden classloader so far seems a better way to go, but for long
>> term, I wish the JSR 277[1] or JSR 291[2] can be considered to address
>> this
>> kind of issue in general.
>
> IIRC JSR277 doesn't have the concept of classloaders, so this wouldn't
> work. It only enforces modularity at compile time, at least, for my
> understanding.
>
> JSR 291 (OSGi) of course solves this problem, and given that Harmony
> is being put together as a suite of bundles, would probably make sense
> to at least emulate. Equninox uses a classloader to do load all the
> main ones and decide whether or not to pass things up to the parent
> classloader. You could then set this as the thread's context
> classloader; but I don't know how that would work for e.g. newly
> created threads.
>
> PS there's a bunch of config options that you can specify to configure
> what the osgi classloader does:
>
> http://wiki.eclipse.org/index.php/FAQ_How_do_I_add_a_library_to_the_classpath_of_a_plug-in%3F
>
>
> "setting the osgi.parentClassloader system property on startup ... to
> one of boot, ext, app or fwk."
>
> These configure what items are visible from the classloader.
There are a number of reasons why using Equinox directly will not work
for Harmony, where we are modularizing the class library itself. Not
least of which is that the very Java types used to implement Equinox
need to be part of the managed bundles. It's an area of ongoing work.
Regards,
Tim
Re: [drlvm] The Return of the Hidden Classloader, Part II
Posted by Alex Blewitt <al...@gmail.com>.
On 07/02/07, Yang Paulex <pa...@gmail.com> wrote:
> 2007/2/7, Geir Magnusson Jr. <ge...@pobox.com>:
> >
> > We never solved the problem of how to hide non java*. packages that
> > are on the boot classpath from apps. We talked about a few
> > possibilities :
> >
> > 2) have a "hidden" classloader that only the system classloader can use.
> >
> > I like #2. Has anyone looked into this or made any progress? Do
> > people think this is as important as I do?
>
> +1, the hidden classloader so far seems a better way to go, but for long
> term, I wish the JSR 277[1] or JSR 291[2] can be considered to address this
> kind of issue in general.
IIRC JSR277 doesn't have the concept of classloaders, so this wouldn't
work. It only enforces modularity at compile time, at least, for my
understanding.
JSR 291 (OSGi) of course solves this problem, and given that Harmony
is being put together as a suite of bundles, would probably make sense
to at least emulate. Equninox uses a classloader to do load all the
main ones and decide whether or not to pass things up to the parent
classloader. You could then set this as the thread's context
classloader; but I don't know how that would work for e.g. newly
created threads.
PS there's a bunch of config options that you can specify to configure
what the osgi classloader does:
http://wiki.eclipse.org/index.php/FAQ_How_do_I_add_a_library_to_the_classpath_of_a_plug-in%3F
"setting the osgi.parentClassloader system property on startup ... to
one of boot, ext, app or fwk."
These configure what items are visible from the classloader.
Alex.
Re: [drlvm] The Return of the Hidden Classloader, Part II
Posted by Yang Paulex <pa...@gmail.com>.
2007/2/7, Geir Magnusson Jr. <ge...@pobox.com>:
>
> This is an oldie but goodie..
>
> We never solved the problem of how to hide non java*. packages that
> are on the boot classpath from apps. We talked about a few
> possibilities :
>
> 1) repackage them to a different package namespace
>
> 2) have a "hidden" classloader that only the system classloader can use.
>
> I like #2. Has anyone looked into this or made any progress? Do
> people think this is as important as I do?
+1, the hidden classloader so far seems a better way to go, but for long
term, I wish the JSR 277[1] or JSR 291[2] can be considered to address this
kind of issue in general.
[1] http://jcp.org/en/jsr/detail?id=277
[2] http://jcp.org/en/jsr/detail?id=291
geir
>
>
--
Paulex Yang
China Software Development laboratory
IBM
Re: [drlvm] The Return of the Hidden Classloader, Part II
Posted by Naveen Neelakantam <ne...@uiuc.edu>.
On Feb 6, 2007, at 10:18 PM, Geir Magnusson Jr. wrote:
> This is an oldie but goodie..
>
> We never solved the problem of how to hide non java*. packages that
> are on the boot classpath from apps. We talked about a few
> possibilities :
>
> 1) repackage them to a different package namespace
>
> 2) have a "hidden" classloader that only the system classloader can
> use.
>
> I like #2. Has anyone looked into this or made any progress? Do
> people think this is as important as I do?\]
I like #2 also, but I haven't thought about what it would take to
implement.
>
> geir
>