Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface

["Geir Magnusson Jr." <ge...@apache.org>]

On Jun 3, 2005, at 5:22 PM, Aaron Hamid wrote:


> I'm sort of confused by this discussion also.
>
> It seems to be about at what granularity we start introducing vm- 
> specific interfaces/implementations.  At some point the class  
> library has to interface with the VM.  We can do that at a very  
> high level of course, by simply subsuming the entirety of  
> java.lang.* and "avoid" this issue by simply making the java.lang  
> package THE "interface",
>

Well, if GNU Classpath is any indication, you *can't* do that w/o  
extending java.lang, and therefore you aren't avoiding the issue -  
you are just burying it in java.lang.


> but we certainly don't want to rewrite and be responsible for all  
> that code if we can avoid it.  The only other alternative (at least  
> that I have read) is some compromise whereby "core" classes rely on  
> an "even-more-core" VM-specific API that will necessarily be  
> different for each VM.
>

Yes - and that would be opaque to the class library  
implementations...  for example, the interface contract would specify  
what is needed for both sides (VM and class lib) and then it's up to  
the implementor of the VM to do the right thing wrt the interface w/  
the full freedom to do it in any way they want, and similar for the  
class library implementation.


> As far as I can tell, there is no such standard interface.  So the  
> best we have is to use the ad-hoc interface required by our most  
> likely class library candidate (Classpath), and simultaneously try  
> to ignite interest in standardizing the aforementioned interface  
> across many VMs.
>

Well, that's one way.  Another might be to start with the GNU  
Classpath interface and work out a "standard" interface based on

a) further needs (i.e. what does 1.5 require?)
b) spec/engineering considerations around java.lang (still to be  
hashed out)
c) the experience of implementors like IBM/Sun/BEA that have done  
this too

I'd be happy to offer a patch to GNU Classpath that supported such a  
beastie if we could ever come up with it.



> Obviously this is not going to gain us any leverage with existing  
> proprietary VM
> s unless they also retrofit their library
>

Well, if we pay attention to what they are doing, we may be able to  
pull this off.  And why wouldn't they want to?  They all are  
reinventing the wheel, a wheel that doesn't provide them with any  
advantages over their competitors - I can't see why they wouldn't  
want to standardize on a common approach and codebase :)


> - the only option in that case is to excise everything but  
> java.lang.* from our bundling of Classpath and try to glue on the  
> remaining portion of, say, Sun's library, or IBM's library; of  
> course then those third party libraries must not cheat and use some  
> of their own unpublished VM-specific interfaces, which they already  
> do.
>

Right - we need to figure out why they do it, what they need, and  
what we can use for our interface.  The point is to stop the madness....


>
> [vm]
> [vm-interface]
> [java.lang.* core (e.g. from Classpath)]
> [rest of the class library (e.g rest of Classpath, or some third  
> party library)]
>
> I would not base any policy on support for developers breaking  
> language rules.  Yes you can "cheat" and use reflection to bypass  
> visibility limitations (and I have even had to do this on some  
> occasions to hack around some things), but you leave compatibility  
> and portability at the door when you start doing such things.
>

Where?  in the VM?  I think the VM can do whatever it wants, as long  
as the code executing on the VM is secure....

geir


>
> Aaron
>
> Tom Tromey wrote:
>
>
>>>>>>> "Dan" == Dan Lydick <dl...@earthlink.net> writes:
>>>>>>>
>>>>>>>
>> Dan> That includes the language protection features like Geir's
>> Dan> example of package private, which are an interesting game of
>> Dan> navigating class file structures with reflection, etc.  I've
>> Dan> never tried that, but Geir is right, I think it can be done if
>> Dan> you try hard enough.
>> There are two cases.
>> In the first case there is no security manager.  All the code is
>> trusted.  In this case, yes, using reflection you can work around
>> access protections.  In practice I don't see how this matters, as the
>> code is already trusted... if it does something weird, well, so what?
>> It could also do weird things to your filesystem or anything else.
>> In the second case, when there is a security manager, then, no, you
>> cannot get around the access controls, even with reflection.
>> Dan> The underlying idea here is to make as few changes as  
>> possible to
>> Dan> as little of the java.*, especially java.lang.*, or other core
>> Dan> library packages in order to give the Harmony JVM runtime
>> Dan> environment the greatest flexibility for using libraries.  Heck,
>> Dan> if it's done right, you might be able to use Sun's or IBM's
>> Dan> java.* library implementation!
>> I see where you're coming from, but I don't see how this furthers the
>> goal of Harmony, which as I understand it is to have a complete,
>> compatible, open J2SE implementation.
>> But then, I'm not really getting this part of the discussion.  For
>> instance, why does Harmony need a VM layer different from the one
>> Classpath provides?  I don't understand that.
>> Tom
>>
>>
>
>
>

-- 
Geir Magnusson Jr                                  +1-203-665-6437
geirm@apache.org

Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface

[Sven de Marothy <sv...@physto.se>]

+1 to that!

On Sun, 2005-06-05 at 11:53 -0400, Aaron wrote:
>  From what I understood from this thread (and of course my understanding 
> could be wrong), there is some contention over where to "hide" this 
> Classlib-VM interface and implementation so that user code is least able 
> to use/abuse it.  One suggestion was to use existing package visibility 
> modifiers and stash the classes in java.lang.  Another was to take these 
> classes and put them in a package other than java.lang to keep java.lang 
> "pure" (at which point they presumably would have to be "public").  
> Another was to use classloader or VM "magic" (or perhaps some more 
> sophisticated module publishing scheme) to hide the existence of these 
> classes.
> 
> In my humble opinion, I'm not overly concerned about spending a lot of 
> effort to "hide" this code from application code, because it is already 
> demonstrably wrong to use them (from user code), and with the proper 
> measures one can easily circumvent the security manager and access 
> hidden fields/methods anyway.
> 
> Frankly, 
> move-forward-with-Classpath-design-and-change-in-the-future-if-we-need-to 
> sounds fine to me.
> 
> Aaron
> 

Re: [arch] How much of java.* and friends does Harmony need to write. Was: VM/Classlibrary interface

[Aaron <ar...@cornell.edu>]

 From what I understood from this thread (and of course my understanding 
could be wrong), there is some contention over where to "hide" this 
Classlib-VM interface and implementation so that user code is least able 
to use/abuse it.  One suggestion was to use existing package visibility 
modifiers and stash the classes in java.lang.  Another was to take these 
classes and put them in a package other than java.lang to keep java.lang 
"pure" (at which point they presumably would have to be "public").  
Another was to use classloader or VM "magic" (or perhaps some more 
sophisticated module publishing scheme) to hide the existence of these 
classes.

In my humble opinion, I'm not overly concerned about spending a lot of 
effort to "hide" this code from application code, because it is already 
demonstrably wrong to use them (from user code), and with the proper 
measures one can easily circumvent the security manager and access 
hidden fields/methods anyway.

Frankly, 
move-forward-with-Classpath-design-and-change-in-the-future-if-we-need-to 
sounds fine to me.

Aaron

Geir Magnusson Jr. wrote:

>
> On Jun 3, 2005, at 5:22 PM, Aaron Hamid wrote:
>
>
>> I'm sort of confused by this discussion also.
>>
>> It seems to be about at what granularity we start introducing vm- 
>> specific interfaces/implementations.  At some point the class  
>> library has to interface with the VM.  We can do that at a very  high 
>> level of course, by simply subsuming the entirety of  java.lang.* and 
>> "avoid" this issue by simply making the java.lang  package THE 
>> "interface",
>>
>
> Well, if GNU Classpath is any indication, you *can't* do that w/o  
> extending java.lang, and therefore you aren't avoiding the issue -  
> you are just burying it in java.lang.
>
>
>> but we certainly don't want to rewrite and be responsible for all  
>> that code if we can avoid it.  The only other alternative (at least  
>> that I have read) is some compromise whereby "core" classes rely on  
>> an "even-more-core" VM-specific API that will necessarily be  
>> different for each VM.
>>
>
> Yes - and that would be opaque to the class library  
> implementations...  for example, the interface contract would specify  
> what is needed for both sides (VM and class lib) and then it's up to  
> the implementor of the VM to do the right thing wrt the interface w/  
> the full freedom to do it in any way they want, and similar for the  
> class library implementation.
>
>
>> As far as I can tell, there is no such standard interface.  So the  
>> best we have is to use the ad-hoc interface required by our most  
>> likely class library candidate (Classpath), and simultaneously try  
>> to ignite interest in standardizing the aforementioned interface  
>> across many VMs.
>>
>
> Well, that's one way.  Another might be to start with the GNU  
> Classpath interface and work out a "standard" interface based on
>
> a) further needs (i.e. what does 1.5 require?)
> b) spec/engineering considerations around java.lang (still to be  
> hashed out)
> c) the experience of implementors like IBM/Sun/BEA that have done  
> this too
>
> I'd be happy to offer a patch to GNU Classpath that supported such a  
> beastie if we could ever come up with it.
>
>
>
>> Obviously this is not going to gain us any leverage with existing  
>> proprietary VM
>> s unless they also retrofit their library
>>
>
> Well, if we pay attention to what they are doing, we may be able to  
> pull this off.  And why wouldn't they want to?  They all are  
> reinventing the wheel, a wheel that doesn't provide them with any  
> advantages over their competitors - I can't see why they wouldn't  
> want to standardize on a common approach and codebase :)
>
>
>> - the only option in that case is to excise everything but  
>> java.lang.* from our bundling of Classpath and try to glue on the  
>> remaining portion of, say, Sun's library, or IBM's library; of  
>> course then those third party libraries must not cheat and use some  
>> of their own unpublished VM-specific interfaces, which they already  do.
>>
>
> Right - we need to figure out why they do it, what they need, and  
> what we can use for our interface.  The point is to stop the madness....
>
>
>>
>> [vm]
>> [vm-interface]
>> [java.lang.* core (e.g. from Classpath)]
>> [rest of the class library (e.g rest of Classpath, or some third  
>> party library)]
>>
>> I would not base any policy on support for developers breaking  
>> language rules.  Yes you can "cheat" and use reflection to bypass  
>> visibility limitations (and I have even had to do this on some  
>> occasions to hack around some things), but you leave compatibility  
>> and portability at the door when you start doing such things.
>>
>
> Where?  in the VM?  I think the VM can do whatever it wants, as long  
> as the code executing on the VM is secure....
>
> geir
>
>
>>
>> Aaron
>>
>> Tom Tromey wrote:
>>
>>
>>>>>>>> "Dan" == Dan Lydick <dl...@earthlink.net> writes:
>>>>>>>>
>>>>>>>>
>>> Dan> That includes the language protection features like Geir's
>>> Dan> example of package private, which are an interesting game of
>>> Dan> navigating class file structures with reflection, etc.  I've
>>> Dan> never tried that, but Geir is right, I think it can be done if
>>> Dan> you try hard enough.
>>> There are two cases.
>>> In the first case there is no security manager.  All the code is
>>> trusted.  In this case, yes, using reflection you can work around
>>> access protections.  In practice I don't see how this matters, as the
>>> code is already trusted... if it does something weird, well, so what?
>>> It could also do weird things to your filesystem or anything else.
>>> In the second case, when there is a security manager, then, no, you
>>> cannot get around the access controls, even with reflection.
>>> Dan> The underlying idea here is to make as few changes as  possible to
>>> Dan> as little of the java.*, especially java.lang.*, or other core
>>> Dan> library packages in order to give the Harmony JVM runtime
>>> Dan> environment the greatest flexibility for using libraries.  Heck,
>>> Dan> if it's done right, you might be able to use Sun's or IBM's
>>> Dan> java.* library implementation!
>>> I see where you're coming from, but I don't see how this furthers the
>>> goal of Harmony, which as I understand it is to have a complete,
>>> compatible, open J2SE implementation.
>>> But then, I'm not really getting this part of the discussion.  For
>>> instance, why does Harmony need a VM layer different from the one
>>> Classpath provides?  I don't understand that.
>>> Tom
>>