You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Tom Tromey <tr...@redhat.com> on 2005/05/09 21:10:45 UTC
State of the World
I've started writing, and then deleted, a few messages to this list
about different aspects of what I see as the solution -- using
Classpath, notes on licenses, etc. But this proved unsatisfactory and
I decided instead to start with a note about what I see as the
problems in the free java space.
First a few non-problems:
* The class library. Classpath already has a ton of code and a
reasonable approach to VM independence. The VM layer still changes
from release to release, but that will quiet down as we move toward
completion.
* Licensing. The intent of the Classpath license was that it be
acceptable for use in proprietary projects. This should probably be
its own thread, but basically, if this isn't acceptable, I think we
can work on it.
* The tool space is likewise fairly full. There are free compilers of
various sorts; also jar, javadoc, rmic, etc.
* Virtual machines (as distinct from execution engines fwiw). There
are a lot of VMs out there, suiting a wide range of needs; kaffe
alone has probably been ported to every conceivable niche twice.
Open problems
* The class library is incomplete. I tend to dump this in the 'bug'
category. Anyway, the reason the class library appears in both the
non-problem and problem lists is that it seems unlikely that a new
project would address this more efficiently than just hacking on
Classpath.
* Fragmentation. I think there are too many free JVMs. In particular
the "C/C++-based VM" niche is over-full. It ought to be possible,
though difficult, to write a configurable core VM that can be reused
by other projects. The idea here is, have a reusable reference
implementation, and reduce writing a VM to writing an execution
engine (and perhaps a GC).
* Parts of testing. While our class library tests are manageable (we
probably have fewer tests than we need, but we have a lot and adding
more is trivial), there are still entire areas (VM level tests for
instance) where there is not even the start of a shared test suite.
This becomes particularly important when you starting looking at,
say, writing a Mozilla plugin and executing untrusted code.
* TCK access. Dalibor is working on this, but this seems like an area
where Apache's insider status and superior connections could really
expedite things.
* J2ME. There has been talk about approaches to supporting both J2ME
and J2SE in Classpath using the same source base, but no real
action. This might be an interesting problem.
* Publicity. Though we try, and though we're starting to see some
successes, it does seem that just the announcement of Harmony has
generated more PR than Classpath got over the last year.
* com.sun.*. Those pesky java programmers often end up using unfree
APIs. I'm not sure what Harmony could do here; maybe pressure Sun
into make it much more difficult to do this?
As a thought experiment, look at the free JVM space from, say, the
gcj/Linux distro perspective. What is left undone? It is basically:
"where is JDWP?" (or whatever lets us debug from inside Eclipse) and
"we need a complete library".
Or, if you're interested in a more hotspot-like execution approach,
look from the JikesRVM angle... I won't speculate here, though, since
I'm not really involved with that project.
Maybe it seems like a grim environment for starting a new project.
Well, considered on technical grounds alone, it is. One needs a
pretty compelling technical story to do better than already existing
projects.
Tom
Re: Isolates support
Posted by Doug Lea <dl...@cs.oswego.edu>.
Francisco Vega wrote:
> Hi everybody,
>
> planning the development of a new virtual machine at this moment, it
> could be interesting to provide support for isolations.
This is probably off-topic, but...
1. The second (and hopefully final) public review draft of JSR121
will be available from http://jcp.org very soon.
2. If approved (it would go to vote sometime this summer), a reference
implementation and TCK will probably be out before the end of year.
3. It is reasonable to expect that Isolates will be a mandatory
part of J2SE as of 1.7 (J2SE7), but optional before then. Not
making it in as mandatory for either 1.5 or 1.6 led to these
hold-states until we decided to rework specs a bit
to make isolates supportable as a non-core package.
-Doug
Re: Platform Support
Posted by "Geir Magnusson Jr." <ge...@apache.org>.
On May 10, 2005, at 8:20 AM, Rivaaj Jumna wrote:
> Hi
>
> The JVM under discussion will obviously support multiple platforms,
> running
> under Linux, Windows, etc. Will it also support other platforms as
> in being
> "modular enough" to be composed to run on Pocket PC or cellphones
> (j2me) or
> is such support outside of the initial scope for this project?
Yes. One of the motivations is to reduce the amount of work someone
wishing to have a compatible version of J2SE on a "new" platform - be
it new hardware, new operating system, or even new kinds of
performance optimizations on existing platforms.
geir
--
Geir Magnusson Jr +1-203-665-6437
geirm@apache.org
Platform Support
Posted by Rivaaj Jumna <ri...@gmail.com>.
Hi
The JVM under discussion will obviously support multiple platforms, running
under Linux, Windows, etc. Will it also support other platforms as in being
"modular enough" to be composed to run on Pocket PC or cellphones (j2me) or
is such support outside of the initial scope for this project?
Thanks
Re: Isolates support
Posted by Dalibor Topic <ro...@kaffe.org>.
Francisco Vega wrote:
> Hi everybody,
>
> planning the development of a new virtual machine at this moment, it
> could be interesting to provide support for isolations.
> I supose that most people will know the concept, but in short, the idea
> is to run several Java programs in parallel in one virtual machine (one
> OS process) and at the same time:
> - keep all the Java programs in separate and effective sandboxes
> - share the maximum resources (decompiled class code,etc)
Hi Francisco,
Yeah, isolation is pretty nice. There is already an implementation of
the APIs as Free Software since 2002, called JanosVM, and available at
http://www.cs.utah.edu/flux/janos/janosvm.html
"Unlike any available virtual machine, the JanosVM supports multiple,
separate processes (called "teams" in JanosVM) within a single VM. Based
on KaffeOS (and thus Kaffe), the JanosVM supports per-team separate
heaps, per-team garbage collection threads, inter-team thread migration,
safe cross-team reference objects, and a spiffy tutorial. Designed to
support asynchronous termination of uncooperative or malicious Java
bytecode applications, the JanosVM provides robust and scalable
multi-process support within a single virtual machine."
Some of that code has been merged back into Kaffe, but the largest chunk
that actually implements isolation remains to be merged in, volunteers
welcome :) Kaffe developer Patrick Tullman is on the JSR committee,
afaik, and Godmar Back, the author of KaffeOS, is active on the
isolation JSR mailing lists as well, afaik.
Kaffe is a bit odd that way, as a lot of exciting things exist in some
Kaffe fork somewhere, be it mixed-mode engines, isolation, efficient
jits or parallelized virtual machines. It's just merging everthing back
in that takes time. :)
cheers,
dalibor topic
Isolates support
Posted by Francisco Vega <fr...@yahoo.es>.
Hi everybody,
planning the development of a new virtual machine at this moment, it
could be interesting to provide support for isolations.
I supose that most people will know the concept, but in short, the idea
is to run several Java programs in parallel in one virtual machine (one
OS process) and at the same time:
- keep all the Java programs in separate and effective sandboxes
- share the maximum resources (decompiled class code,etc)
For an indeep discusion of benefits and a running example you can look
at http://research.sun.com/projects/barcelona/mvm/
There is also a JSR in process trying to determine a common basis for
this kind of functionality: JSR-121. In any case, it is not necessary to
comply to this JSR, because it is in some 'hold' state, but the ability
to run isolates is a lower layer that can be implemented without any
kind of JSR commitment.
Regards.
Re: Class Library Modularity [Was Re: State of the World]
Posted by "Richard S. Hall" <he...@ungoverned.org>.
Ricky Clarkson wrote:
>I might have misunderstood this thread, but it seems to have some
>relation to ivy, a dependency manager, see
>http://www.jayasoft.fr/org/modules/ivy/overview.php
>
>
Yes, this IVY does sound like it is related, but not quite the same
thing. IVY sounds like a component repository that you can use in your
own projects to describe your project's dependency on other projects. In
this case, IVY performs a build-time activity.
It appears that IVY allows you to specify dependencies on other JAR
files and then ensures that these JAR files are present when you build.
Without investigating it explicitly, how would IVY allow you to describe
what is exported from each of the individual JAR files (i.e., their
public API) and how would it enforce this visibility at run time?
I actually have a similar tool (but much simpler) to IVY, called OBR,
that performs deployment-time dependency management for package
dependencies for OSGi bundles. OBR is interesting, but it would not be
as interesting without the OSGi run time underneath of it.
So while tools like IVY and OBR are related to the issue, they are just
a part of it.
-> richard
Re: Class Library Modularity [Was Re: State of the World]
Posted by Ricky Clarkson <ri...@gmail.com>.
I might have misunderstood this thread, but it seems to have some
relation to ivy, a dependency manager, see
http://www.jayasoft.fr/org/modules/ivy/overview.php
On 11/05/05, Richard S. Hall <he...@ungoverned.org> wrote:
> Peter Donald wrote:
>
> > Richard S. Hall wrote:
> >
> >> However, while I agree that Sun's implementation does provide some
> >> modularity mechanisms, there seems to be two short comings:
> >>
> >> 1. The mechanisms themselves do not go far enough, they only provide
> >> minimal capabilities.
> >
> >
> > True but what more do you need at the runtime level?
>
> Well, I can think of one thing I would like, some way for the packages
> contained in a JAR file be able to inform the underlying runtime not
> only its external dependencies, but also what it provides (or what it
> exposes). Due to limitations in the Java visibility/protection rules,
> sometimes it is not possible to avoid making implementation API public
> because you are forced to declare classes/methods as public if you need
> to use them from more than one package.
>
> Of course, you could just put everything in the same package and use
> package protection, but that also defeats the purpose of trying to
> structure things into modules. So, for a module to be able to say that I
> export package "foo", but not "foo.FooImpl" would be very useful from my
> perspective. Then other packages inside the JAR are privy to
> implementation APIs, where packages outside the JAR are not.
>
> I also have lots of interesting ideas about dynamic deployment/updates
> too, but I am sure that these things would be extension to the platform
> as opposed to implementing a conforming J2SE platform. :-)
>
> -> richard
>
>
Re: Class Library Modularity [Was Re: State of the World]
Posted by "Richard S. Hall" <he...@ungoverned.org>.
Peter Donald wrote:
> What you are asking for here is new language extensions - effectively
> an "assembly" or "jar" access specifier to go along with private,
> protected, public and package access. This would be very useful -
> particularly if different jars/assemblies were of higher importance in
> runtime. However it is not part of the Java standard and thus I
> suspect that best way to achieve this goal is go through the jcp process.
Well, I think that this is certainly one approach to implementing what I
am talking about, but certainly not the only way. Perhaps not even the
best way. Maybe it is better to have such module constructs outside the
language and part of the deployment/package layer. I don't know.
> You *could* implement this in Suns java by writing a custom
> ClassLoader and using Annotations but ... unless it is part of a
> standard somewhere it is unlikely to get broad adoption.
Well, this is exactly the kind of stuff that is being done with the OSGi
framework. So that is sort of where I am coming from.
> You can implement this in suns java. Just write your own ClassLoader
> to "hide" specified classes, set the ClassLoader as the bootstrap
> classloader and it is done.
Yes, this is basically the approach. Unfortunately, there are lots of
little "gotchas" in here, because if you hide the wrong classes, then
you end up getting Sun's JVM throwing exceptions at runtime because it
cannot find "sun.*" classes, for example.
It is hard to write code on Sun's JVM that is isolated from Sun's
private API because there is an implicit assumption that every class
loader leads back to Sun's impl classes and this assumption is required
by Sun's libraries to function. This is not true of all JVMs however.
> I would love get my hands on the platform to implement Isolate and
> Resource Managers as in Suns MVM (research JVM) as that would make
> writing scalable applications in java so much easier.
I agree. I think these technologies are looking really interesting.
-> richard
Re: Class Library Modularity [Was Re: State of the World]
Posted by Peter Donald <pe...@realityforge.org>.
Richard S. Hall wrote:
> Well, I can think of one thing I would like, some way for the packages
> contained in a JAR file be able to inform the underlying runtime not
> only its external dependencies, but also what it provides (or what it
> exposes). Due to limitations in the Java visibility/protection rules,
> sometimes it is not possible to avoid making implementation API public
> because you are forced to declare classes/methods as public if you need
> to use them from more than one package.
What you are asking for here is new language extensions - effectively an
"assembly" or "jar" access specifier to go along with private,
protected, public and package access. This would be very useful -
particularly if different jars/assemblies were of higher importance in
runtime. However it is not part of the Java standard and thus I suspect
that best way to achieve this goal is go through the jcp process.
You *could* implement this in Suns java by writing a custom ClassLoader
and using Annotations but ... unless it is part of a standard somewhere
it is unlikely to get broad adoption.
> Of course, you could just put everything in the same package and use
> package protection, but that also defeats the purpose of trying to
> structure things into modules. So, for a module to be able to say that I
> export package "foo", but not "foo.FooImpl" would be very useful from my
> perspective. Then other packages inside the JAR are privy to
> implementation APIs, where packages outside the JAR are not.
You can implement this in suns java. Just write your own ClassLoader to
"hide" specified classes, set the ClassLoader as the bootstrap
classloader and it is done.
> I also have lots of interesting ideas about dynamic deployment/updates
> too, but I am sure that these things would be extension to the platform
> as opposed to implementing a conforming J2SE platform. :-)
Fun parts ;)
I would love get my hands on the platform to implement Isolate and
Resource Managers as in Suns MVM (research JVM) as that would make
writing scalable applications in java so much easier.
Re: Class Library Modularity [Was Re: State of the World]
Posted by "Richard S. Hall" <he...@ungoverned.org>.
Dalibor Topic wrote:
> I think that would be pretty interesting, actually. One of the hard
> challenges so far for me has been efficient support for 'profiles'[1]
> in class libraries. What I have done in Kaffe is just a simple
> mechanism to select the classes to build as part of a profile, but
> that usually drags in a superset of an API due to implementation
> internals required by added methods in later versions of a class,
> rather then a minimal implementaiton of just the classes, methods,
> fields, and their transitive closure.
>
> My experiments with kaffe last year have shown me that it is possible
> to separate the class library into large chunks with certain ease,
> though unfortunately the maintenace of such a modular seperation is
> burdensome without tool support. From my cursory glance at the OSGi
> specs, it seems as the OSGi devs have put a good amount of though into
> dealing with the subject, so I'd be very interested to hear more about
> it, including about respective tools. :)
Yes, this is definitely something that could benefit from such a
packaging mechanism. If it were possible to chunk the class libraries
into modules, then without some mechanism to support/enforce this
encapsulation the exercise becomes somewhat pointless.
Some of the features I am talking about are already available in the
OSGi spec, but these ideas in general are definitely in the purview of
the OSGi framework and the work done around it. I have an experimental
release of Oscar that starts to deal with the issues we are discussing
here...these things are of interest to the OSGi Alliance Core Platform
Expert Group too.
I agree that these types of mechanisms require some sort of tooling
support. One area that I want to investigate more is the generation of
the module metadata by analyzing its byte code. There are already tools
that do this for the R3 OSGi spec, so it should be possible to extend
the approach to handle the metadata required for this type of
encapsulation stuff too.
Unfortunately, the tool support is still coming up from behind. Maybe
this will change now that Eclipse depends on OSGi as well.
> Yes, I agree. To give an example, an unfortunately common mispractice
> that leads to unportable source code written in the Java programming
> language is the abuse of runtime-specific classes by non-runtime code
> to get access to runtime-specific services. Being able to, with
> relative ease, make sure that modules of a class library only export
> the interfaces specified in the specs would be pretty useful to help
> people avoid writing implementation-specific code.
Exactly. This my thought too. Plus it ensures that the VM and standard
libraries themselves do not do practices that make it difficult to do
various deployment/management activities on top of the VM.
> Well, I am all ears. :) It would surely be interesting to be able
> dynamically update a runtime installation's class libraries, for
> example, or just it's JIT, if one can get the architecture right.
Yes, these are the sort of thoughts I was having too as other people
were having discussions about modular VM architectures.
> On a side note, I'd love to see such mechanisms work hand-in-hand with
> the native platform's package management, to avoid having to jump
> through multiple hoops to manage software.
This is another valid issue, but one that I have not given that much
thought to at this point.
> p.s. OSCAR already works on Kaffe & other GNU Classpath runtimes,
> right? I remember seing some mails from that direction :)
I am aware of users using Oscar on Kaffe, yes. I try to keep it pretty
neutral and try to fix issues if I get reports of difficulties.
-> richard
Re: Class Library Modularity [Was Re: State of the World]
Posted by Dalibor Topic <ro...@kaffe.org>.
Richard S. Hall wrote:
> Peter Donald wrote:
>
>> Richard S. Hall wrote:
>>
>>> However, while I agree that Sun's implementation does provide some
>>> modularity mechanisms, there seems to be two short comings:
>>>
>>> 1. The mechanisms themselves do not go far enough, they only provide
>>> minimal capabilities.
>>
>>
>>
>> True but what more do you need at the runtime level?
>
>
>
> Well, I can think of one thing I would like, some way for the packages
> contained in a JAR file be able to inform the underlying runtime not
> only its external dependencies, but also what it provides (or what it
> exposes). Due to limitations in the Java visibility/protection rules,
> sometimes it is not possible to avoid making implementation API public
> because you are forced to declare classes/methods as public if you need
> to use them from more than one package.
I think that would be pretty interesting, actually. One of the hard
challenges so far for me has been efficient support for 'profiles'[1] in
class libraries. What I have done in Kaffe is just a simple mechanism to
select the classes to build as part of a profile, but that usually drags
in a superset of an API due to implementation internals required by
added methods in later versions of a class, rather then a minimal
implementaiton of just the classes, methods, fields, and their
transitive closure.
My experiments with kaffe last year have shown me that it is possible to
separate the class library into large chunks with certain ease, though
unfortunately the maintenace of such a modular seperation is burdensome
without tool support. From my cursory glance at the OSGi specs, it seems
as the OSGi devs have put a good amount of though into dealing with the
subject, so I'd be very interested to hear more about it, including
about respective tools. :)
> Of course, you could just put everything in the same package and use
> package protection, but that also defeats the purpose of trying to
> structure things into modules. So, for a module to be able to say that I
> export package "foo", but not "foo.FooImpl" would be very useful from my
> perspective. Then other packages inside the JAR are privy to
> implementation APIs, where packages outside the JAR are not.
Yes, I agree. To give an example, an unfortunately common mispractice
that leads to unportable source code written in the Java programming
language is the abuse of runtime-specific classes by non-runtime code to
get access to runtime-specific services. Being able to, with relative
ease, make sure that modules of a class library only export the
interfaces specified in the specs would be pretty useful to help people
avoid writing implementation-specific code.
> I also have lots of interesting ideas about dynamic deployment/updates
> too, but I am sure that these things would be extension to the platform
> as opposed to implementing a conforming J2SE platform. :-)
Well, I am all ears. :) It would surely be interesting to be able
dynamically update a runtime installation's class libraries, for
example, or just it's JIT, if one can get the architecture right.
On a side note, I'd love to see such mechanisms work hand-in-hand with
the native platform's package management, to avoid having to jump
through multiple hoops to manage software.
cheers,
dalibor topic
p.s. OSCAR already works on Kaffe & other GNU Classpath runtimes, right?
I remember seing some mails from that direction :)
[1] Say 1.5, 1.6, Personal Java, ...
Re: Class Library Modularity [Was Re: State of the World]
Posted by "Richard S. Hall" <he...@ungoverned.org>.
Humberto S. N. dos Anjos wrote:
>Just to add a little note: C# (I'm not sure about the other .NET languages)
>deals with this issue by adding a new access modifier, internal, that makes
>the class/method/field/property visible by all classes in the same assembly.
>Could this behavior be emulated using solely the manifest file?
>
>
Interesting fact to know. Thanks.
This behavior can be partially implemented with manifest metadata.
Without giving it much thought, I would say that it wouldn't be possible
for the smaller granularity items (e.g., method, field, property), only
for classes.
Check back with me in a couple weeks, because when I find the time, this
type of class hiding is something that I will have implemented in my
OSGi implementation...I just need to find the time to do it. :-)
-> richard
Re: Class Library Modularity [Was Re: State of the World]
Posted by "Humberto S. N. dos Anjos" <h....@gmail.com>.
> > Woudln't it be great if this was addressed at the language level?
> > Something like declaring "friend" packages or something like that.
> > Then the public modifier would basically fulfil the need for this kind
> > of "provides" or "exposes" declaration?
Just to add a little note: C# (I'm not sure about the other .NET languages)
deals with this issue by adding a new access modifier, internal, that makes
the class/method/field/property visible by all classes in the same assembly.
Could this behavior be emulated using solely the manifest file?
_______________________
Humberto S. N. dos Anjos
Re: Class Library Modularity [Was Re: State of the World]
Posted by "Richard S. Hall" <he...@ungoverned.org>.
Dmitry Serebrennikov wrote:
>
>>
>> Well, I can think of one thing I would like, some way for the
>> packages contained in a JAR file be able to inform the underlying
>> runtime not only its external dependencies, but also what it provides
>> (or what it exposes). Due to limitations in the Java
>> visibility/protection rules, sometimes it is not possible to avoid
>> making implementation API public because you are forced to declare
>> classes/methods as public if you need to use them from more than one
>> package.
>
>
> Woudln't it be great if this was addressed at the language level?
> Something like declaring "friend" packages or something like that.
> Then the public modifier would basically fulfil the need for this kind
> of "provides" or "exposes" declaration?
This could certainly be one way to implement it, but I don't think we
are going to extend the JVM.
> Well, maybe not. If you package a few packages together into a new
> library, each package might have its public interface, but that
> doesn't mean you want those methods as part of your library interface
> too. So the need for "provides" declaration goes beyond the package
> visibility related limitations it seems.
Yes, there are subtle issues here. The benefit of using a JAR
manifest-based approach is that it works with legacy code and does allow
for easier composition of third-party libraries. In theory it may sound
like a good idea to really lock a module down so that outsiders cannot
access its internals, but maybe in reality it would not be good to
completely lock it down. For example, by using a JAR manifest approach,
it is possible for me to develop some extension to a third-party library
that uses its non-public API to improve performance. I could then
package this as a module to properly encapsulate it again. In this way,
I accept the responsibility of using non-public API, but we can still
protect others from accidentally using it.
So, perhaps having two views of a JAR file, one as just a library and
the other as a module, is more flexible. Just a thought.
> Doesn't Eclipse do something like this?
I don't know for sure, but I have heard reports that Eclipse plugins
expose all of their internals.
-> richard
Re: Class Library Modularity [Was Re: State of the World]
Posted by Dmitry Serebrennikov <dm...@earthlink.net>.
>
> Well, I can think of one thing I would like, some way for the packages
> contained in a JAR file be able to inform the underlying runtime not
> only its external dependencies, but also what it provides (or what it
> exposes). Due to limitations in the Java visibility/protection rules,
> sometimes it is not possible to avoid making implementation API public
> because you are forced to declare classes/methods as public if you
> need to use them from more than one package.
Woudln't it be great if this was addressed at the language level?
Something like declaring "friend" packages or something like that. Then
the public modifier would basically fulfil the need for this kind of
"provides" or "exposes" declaration?
Well, maybe not. If you package a few packages together into a new
library, each package might have its public interface, but that doesn't
mean you want those methods as part of your library interface too. So
the need for "provides" declaration goes beyond the package visibility
related limitations it seems.
Doesn't Eclipse do something like this?
-dmitry
Re: Class Library Modularity [Was Re: State of the World]
Posted by "Richard S. Hall" <he...@ungoverned.org>.
Peter Donald wrote:
> Richard S. Hall wrote:
>
>> However, while I agree that Sun's implementation does provide some
>> modularity mechanisms, there seems to be two short comings:
>>
>> 1. The mechanisms themselves do not go far enough, they only provide
>> minimal capabilities.
>
>
> True but what more do you need at the runtime level?
Well, I can think of one thing I would like, some way for the packages
contained in a JAR file be able to inform the underlying runtime not
only its external dependencies, but also what it provides (or what it
exposes). Due to limitations in the Java visibility/protection rules,
sometimes it is not possible to avoid making implementation API public
because you are forced to declare classes/methods as public if you need
to use them from more than one package.
Of course, you could just put everything in the same package and use
package protection, but that also defeats the purpose of trying to
structure things into modules. So, for a module to be able to say that I
export package "foo", but not "foo.FooImpl" would be very useful from my
perspective. Then other packages inside the JAR are privy to
implementation APIs, where packages outside the JAR are not.
I also have lots of interesting ideas about dynamic deployment/updates
too, but I am sure that these things would be extension to the platform
as opposed to implementing a conforming J2SE platform. :-)
-> richard
Re: Class Library Modularity [Was Re: State of the World]
Posted by Peter Donald <pe...@realityforge.org>.
Richard S. Hall wrote:
> However, while I agree that Sun's implementation does provide some
> modularity mechanisms, there seems to be two short comings:
>
> 1. The mechanisms themselves do not go far enough, they only provide
> minimal capabilities.
True but what more do you need at the runtime level?
> 2. They are not used, nor is their use really encouraged.
They are used by a few container APIs (ie mandated in servlet spec and
it was proposed for EJB - not sure if it made it) but because there is
no real support in base runtime no one bothered to use them in regular apps.
> I won't deny that I have a biased point of view, but your point about
> the spaghetti code of the standard runtime is what I am trying to get
> at. It would be nice if Harmony defined some form of modularity
> (preferably a stronger form) and then followed the rules imposed by that
> modularity from the beginning to avoid getting into the spaghetti
> situation.
agreed. But I think that this spagetti-ness stems from the way it is
specified and is not really open to much modification in runtimes - I
could be wrong though.
> I, for one, though, think it is
> possible to do better.
I am sure it is - but someone has to sit down and figure out how ;)
---
Cheers,
Peter Donald
"The power of accurate observation is frequently
called cynicism by those who don't have it."
- George Bernard Shaw (1856-1950)
Re: Class Library Modularity [Was Re: State of the World]
Posted by "Richard S. Hall" <he...@ungoverned.org>.
Peter Donald wrote:
> Java already has the ability to define "Optional Packages"/Extensions
> that have similar features to OSGis bundles. For an overview check out
>
> http://java.sun.com/j2se/1.3/docs/guide/extensions/versioning.html
>
> You can declare the id/version/vendor for both the specification and
> implementation of a module. You can also declare dependencies on other
> modules. However given the spagetti that the standard runtime is it
> may not be entirely possible to do in a clean manner.
Yes.
However, while I agree that Sun's implementation does provide some
modularity mechanisms, there seems to be two short comings:
1. The mechanisms themselves do not go far enough, they only provide
minimal capabilities.
2. They are not used, nor is their use really encouraged.
I won't deny that I have a biased point of view, but your point about
the spaghetti code of the standard runtime is what I am trying to get
at. It would be nice if Harmony defined some form of modularity
(preferably a stronger form) and then followed the rules imposed by that
modularity from the beginning to avoid getting into the spaghetti situation.
I am not necessarily saying the OSGi framework should be used directly
(however, I do think that an OSGi-like layer on top of the JVM would
make Java much more appealing when compared to .NET/Assemblies/GAC), I
just hope that these ideas are considered from the beginning and not
tacked on in some web page somewhere, which no one ever reads...like the
above link.
So, if the decision is that existing modularity mechanisms are good
enough, that's fine...just use 'em. I, for one, though, think it is
possible to do better.
-> richard
Re: Class Library Modularity [Was Re: State of the World]
Posted by Peter Donald <pe...@realityforge.org>.
Hi,
Richard S. Hall wrote:
> Just to raise the point about class library modularity again, since it
> is related to this comment. Access [and assumptions about access] to
> these types of classes is problematic when trying to implement
> stronger Java modularity mechanisms, such as is the case with the OSGi
> framework.
Java already has the ability to define "Optional Packages"/Extensions
that have similar features to OSGis bundles. For an overview check out
http://java.sun.com/j2se/1.3/docs/guide/extensions/versioning.html
You can declare the id/version/vendor for both the specification and
implementation of a module. You can also declare dependencies on other
modules. However given the spagetti that the standard runtime is it may
not be entirely possible to do in a clean manner.
--
Cheers,
Peter Donald
"Never interrupt your enemy when he is making
a mistake." - Napoleon Bonaparte (1769-1821)
Class Library Modularity [Was Re: State of the World]
Posted by "Richard S. Hall" <he...@ungoverned.org>.
Tom Tromey wrote:
>* com.sun.*. Those pesky java programmers often end up using unfree
> APIs. I'm not sure what Harmony could do here; maybe pressure Sun
> into make it much more difficult to do this?
>
Just to raise the point about class library modularity again, since it
is related to this comment. Access [and assumptions about access] to
these types of classes is problematic when trying to implement stronger
Java modularity mechanisms, such as is the case with the OSGi framework.
If it is possible to adopt a stronger modularity model, then it is
possible to eliminate these types of issues. As an example, libraries
packaged for OSGi in a bundle JAR file must explicitly list their
package import/export dependencies; thus, it is trivial for them to
ensure that implementation classes are not visible to the outside world.
It would be nice to see all of the libraries accompanying a JVM be more
module-like, with dependencies and contributions explicitly listed,
making it possible to statically and dynamically verify and deploy
various configurations of a Java platform.
I do not claim to be a Java spec expert, so I do not know what aspects
of these ideas are covered by the spec, but I do know that certain
assumptions made by the Sun implementation make it more difficult to
achieve module isolation (such as the assumption that JVM implementation
classes like sun.* and com.sun.* are available from any class loader,
including application class loaders, which results in the issue raised
above by Tom Tromey). If these things can be improved while remaining
J2SE compliant, then it is a step in the right direction.
-> richard
Re: State of the World (it's security)
Posted by Anthony Green <gr...@redhat.com>.
On Mon, 2005-05-09 at 21:59 -0400, Bob wrote:
> If you wish to develop a
> protocol in which (potentially malicious) code can be WRITTEN on
> computer A, then SENT to computer B and run in a sandbox in a secure
> and flexible manner as determined by the administrator of System B
> ----
> then there is NO open source project that will support that
> functionality, nor is there currently any open source project that
> intends to build that functionality. Not GCJ. Not Kaffee.
We intend to have this functionality in GCJ, and much has already been
done this past year to get us closer to this goal (specifically, the
ability to run gcjwebplugin safely
( http://www.nongnu.org/gcjwebplugin/ )).
AG
Re: State of the World (it's security)
Posted by Anthony Green <gr...@redhat.com>.
On Tue, 2005-05-10 at 21:43 -0400, Bob wrote:
> If Harmony security works only for applets, then it will not be useful
> for my needs.
GNU Classpath implements java.lang.SecurityManager and related code
which may be used in any application. I think we're just left with
simple bugs to fix as opposed to big wide gaping holes with no plans to
fill them.
> Moreover, it needs to be able to securely run Java code
> that wasn't specifically made for Harmony (i.e. the Java spec needs to
> be followed).
That's what's been done.
AG
Re: State of the World (it's security)
Posted by Mark Wielaard <ma...@klomp.org>.
Hi,
On Tue, 2005-05-10 at 21:43 -0400, Bob wrote:
> > You might be able to expand this idea to arbitrary programs run
> within the
> > runtime. I haven't investigated yet.
>
> If Harmony security works only for applets, then it will not be useful
> for my needs. Moreover, it needs to be able to securely run Java code
> that wasn't specifically made for Harmony (i.e. the Java spec needs to
> be followed).
This would obviously be in addition to the normal permission model used.
Which as Anthony explained is already in place in GNU Classpath. It is
just a suggestion for a fun project (imho).
Cheers,
Mark
Re: State of the World (it's security)
Posted by Bob <ci...@earthlink.net>.
> You might be able to expand this idea to arbitrary programs run
within the
> runtime. I haven't investigated yet.
If Harmony security works only for applets, then it will not be useful
for my needs. Moreover, it needs to be able to securely run Java code
that wasn't specifically made for Harmony (i.e. the Java spec needs to
be followed).
Re: State of the World (it's security)
Posted by Mark Wielaard <ma...@klomp.org>.
Hi,
On Tue, 2005-05-10 at 08:47 -0400, Bob wrote:
> > Other interesting approaches are [..] Integrate
> > gcjwebplugin
> > out of process with SELinux mandatory access controls when run from
> > mozilla.
>
> I'm somewhat confused here. Will this security implementation only
> work for applets being run from within Mozilla, or are the more general
> Java 2 Security Model ideas being implemented, available to all Java
> programs?
This was just a suggestion to look into if you want an interesting
challenge. I believe really sandboxing the gcjwebplugin runtime spawned
for mozilla with SELinux access controls will provide an extra layer of
security that is even harder to escape from by malicious code. You might
be able to expand this idea to arbitrary programs run within the
runtime. I haven't investigated yet.
Cheers,
Mark
Re: State of the World (it's security)
Posted by Bob <ci...@earthlink.net>.
> Please do some research before posting...
I'm glad to learn about all the progress that's being made. My
comments are based on correspondence I had last summer with the FSF.
At that time, there were no compunctions against criticizing me for
"falling into the Java trap", and yet no expressed interest in building
a free implementation of the security model either. The attitude was
"if your program can't run on GCJ, it's not worth writing." As of last
summer, Java was the ONLY system that could do what I needed on Linux.
Also, as you point out, the security implementations you describe have
made encouraging progress, but have not reached a trustworthy state.
> Classpath and report (and fix!) any issues found. Integrate
> gcjwebplugin
> out of process with SELinux mandatory access controls when run from
> mozilla.
I'm somewhat confused here. Will this security implementation only
work for applets being run from within Mozilla, or are the more general
Java 2 Security Model ideas being implemented, available to all Java
programs?
== Bob
Re: State of the World (it's security)
Posted by Mark Wielaard <ma...@klomp.org>.
Hi,
On Mon, 2005-05-09 at 21:59 -0400, Bob wrote:
> >> Maybe it seems like a grim environment for starting a new project.
> As I mentioned in my last post, proper implementation (and
> verification) of the Java Security Model is a strongly compelling
> technical story. And it is still WIDE OPEN. If you wish to develop a
> protocol in which (potentially malicious) code can be WRITTEN on
> computer A, then SENT to computer B and run in a sandbox in a secure
> and flexible manner as determined by the administrator of System B ----
> then there is NO open source project that will support that
> functionality, nor is there currently any open source project that
> intends to build that functionality. Not GCJ. Not Kaffee. Not .GNU.
Please do some research before posting. GNU Classpath provides all the
packages, classes and code needed by the core libraries to support this.
The various runtimes (ikvm, gcj and kaffe) all come with a byte code
verifier. There is also a Mauve verify module containing verifier tests.
What is missing is a jarsigner utility (but jar verification works,
including assigning ProtectionDomains) and there certainly some bugs
(see gcc bugzilla to find some) since it hasn't been stress tested. This
is the reason gcjwebplugin is currently blocked from entering Debian and
Fedora, they (rightly!) think it is currently to high a security risk.
What needs to be done to complete all this is write more testcases for
the Mauve verifier component. Audit all code. And actually fix any
issues found. Probably the best way to do this is by getting a copy of
Inside Java 2 Architecture, API Design and Implementation from Li Gong
(get the second edition!) and go through it and the GNU Classpath
library code to make sure all permission checks are done at the right
places.
Other interesting approaches are using valgrind to find low level bugs.
Or run tools like FindBugs, CheckStyle, JLint, PMD, etc over GNU
Classpath and report (and fix!) any issues found. Integrate gcjwebplugin
out of process with SELinux mandatory access controls when run from
mozilla.
Cheers,
Mark
Re: State of the World (it's security)
Posted by Bob <ci...@earthlink.net>.
>> Maybe it seems like a grim environment for starting a new project.
>> Well, considered on technical grounds alone, it is. One needs a
>> pretty compelling technical story to do better than already existing
>> projects.
As I mentioned in my last post, proper implementation (and
verification) of the Java Security Model is a strongly compelling
technical story. And it is still WIDE OPEN. If you wish to develop a
protocol in which (potentially malicious) code can be WRITTEN on
computer A, then SENT to computer B and run in a sandbox in a secure
and flexible manner as determined by the administrator of System B ----
then there is NO open source project that will support that
functionality, nor is there currently any open source project that
intends to build that functionality. Not GCJ. Not Kaffee. Not .GNU.
Not Mono (although Mono seems to be the closest, maybe).
The Palladium proposal, to the extent it is still alive, should give
extra impetus for this project. Microsoft has argued that Palladium
will make your computer "more secure" against viruses. This is FUD
designed to scare people into "trusted computing". Well, so will Java
and Java-like systems, and they'll do it without wresting control of
the computer from the end user. Of course, only commercial Java
systems do it today. But imagine the potential a secure Java or
Java-like system could have, if it were easily integratable into a wide
range of OS and desktop applications.
-- Bob
Re: State of the World
Posted by Henri Yandell <fl...@gmail.com>.
On 09 May 2005 13:10:45 -0600, Tom Tromey <tr...@redhat.com> wrote:
>
> Open problems
>
> * The class library is incomplete. I tend to dump this in the 'bug'
> category. Anyway, the reason the class library appears in both the
> non-problem and problem lists is that it seems unlikely that a new
> project would address this more efficiently than just hacking on
> Classpath.
Just to add one.
API Versions. Harmony needs to get 1.5 implemented, with an eye on 1.6.
> * Fragmentation. I think there are too many free JVMs. In particular
> the "C/C++-based VM" niche is over-full. It ought to be possible,
> though difficult, to write a configurable core VM that can be reused
> by other projects. The idea here is, have a reusable reference
> implementation, and reduce writing a VM to writing an execution
> engine (and perhaps a GC).
Versions comes in again. Long term, will there have multiple VM spec
implementations?
Another aside. Using 1.3 as an example; it would have been nice to
have been able to compile Harmony 1.2 and compile in the 1.3 regex
library, rather than moving wholesale to 1.3.
I know there's a sf.net project for adding the 1.5 libraries to 1.4.
Maybe that would just be the easier solution; relatively standalone
apis like collections, regexp, logging, even nio perhaps could be
available as separate jars. These are really just examples; I mainly
mean similar styles of libraries in 1.6 etc.
> * Publicity. Though we try, and though we're starting to see some
> successes, it does seem that just the announcement of Harmony has
> generated more PR than Classpath got over the last year.
>
> * com.sun.*. Those pesky java programmers often end up using unfree
> APIs. I'm not sure what Harmony could do here; maybe pressure Sun
> into make it much more difficult to do this?
Offer similar functionality from Harmony?
My big wish on the tools list is that they be APIs as well as command
line tools. A definite area for vendor improvements.
> Maybe it seems like a grim environment for starting a new project.
> Well, considered on technical grounds alone, it is. One needs a
> pretty compelling technical story to do better than already existing
> projects.
I think you've hit on some of the major parts.
Harmony can bring various existing bits together, underline to
open-source coders as a whole that it's not just a GPL product, break
'open-java' into the corporates, help strengthen JCP backing, and
offer a large user base. Apache hasn't got that many potential VM
coders (I think), but does have a lot of VM users and test-writers.
Hen
Re: State of the World
Posted by Tom Tromey <tr...@redhat.com>.
>>>>> "Ben" == Ben Laurie <be...@algroup.co.uk> writes:
Ben> Tom Tromey wrote:
>> * Fragmentation. I think there are too many free JVMs. In particular
>> the "C/C++-based VM" niche is over-full. It ought to be possible,
>> though difficult, to write a configurable core VM that can be reused
>> by other projects. The idea here is, have a reusable reference
>> implementation, and reduce writing a VM to writing an execution
>> engine (and perhaps a GC).
Ben> This is certainly my understanding of the "modularity" aim, and one of
Ben> my two main interests in this project (that and security, natch).
Ben> Like they said, pointers to where you think the natural fault-lines in
Ben> a JVM are would be most illuminating.
A few places where fault lines occur:
* OS portability layer. This one is pretty obvious.
* Architecture portability layer. This can range from very small
(jamvm has some small amount of assembly for atomic operations) to
medium (e.g., using libffi to enable calls to C functions), to very
large (porting a JIT back end).
* Execution engine choice. This is actually not uncommon as a
configurability tweak. Some VMs come with "slow" and "fast"
compilers; some, like kaffe, treat this as a compile-time choice,
where you compile in the interpreter or jit you want; with gcj you
can mix and match pre-compiled code with interpreted code (though on
some of the less common platforms the interpreter is not available).
* Garbage collector. A few VMs have experimented in this area.
libgcj has a very primitive approach here, since it is basically
tied to a conservative GC (but we have had at least one case where
someone wrote their own replacement GC and substituted it).
JikesRVM and ORP both have pluggable/reconfigurable GCs of some
kind. My recollection from the ORP sources is that the JITs would
call into the GC to find information about things like write
barriers and whatnot.
You can drill down a bit farther and look at a lot of ABI-type issues,
like object layout, synchronization approach, exception handling
approach, parameter passing, etc. In many cases there's no particular
reason you have to definitively make the final decision up front.
Instead, you can do things modularly and use inlining or similar
things to remain efficient. What I'm trying to say here is that, of
course you have to make concrete decisions when writing the VM --
otherwise nothing gets done. But, you don't generally need to spread
these decisions throughout the code.
Tom
Re: State of the World
Posted by Archie Cobbs <ar...@dellroad.org>.
Ben Laurie wrote:
>> * Fragmentation. I think there are too many free JVMs. In particular
>> the "C/C++-based VM" niche is over-full. It ought to be possible,
>> though difficult, to write a configurable core VM that can be reused
>> by other projects. The idea here is, have a reusable reference
>> implementation, and reduce writing a VM to writing an execution
>> engine (and perhaps a GC).
>
> This is certainly my understanding of the "modularity" aim, and one of
> my two main interests in this project (that and security, natch).
>
> Like they said, pointers to where you think the natural fault-lines in a
> JVM are would be most illuminating.
Hi all,
I'm the author of JCVM [1] and a contributor to Classpath. Some quick
thoughts...
Harmony sounds like a good idea overall and I like it. IMHO the more
free software and options for people to play with and improve the
better.
As for the above idea of a "modularized" VM core, I'm a bit skeptical.
To me it's a bit like saying, "Why are there so many open source UNIX
kernels? Why isn't there a single modulurized kernel on which the Linux,
FreeBSD, NetBSD, etc. kernels are based?"
In other words, for something as tightly entwined as a Java VM, there
are invariable compromises when you modularize things. Sure you can
write APIs for the execution engine and the garbage collector, but
that's going to seriously hamper the ability to do the coolest kind
of optimizations (and performance is a goal here, right?)
Not saying it's a bad idea -- rather, it's a neat idea and a very
interesting challenge technically. However, who is going to do the
work? The existing VM authors are not going to be interested in
moving their code to this modular core when the ultimate result is
simply a slower version of their existing VM. So probably what will
happen is that yet another Java VM will be written basically from
scratch -- not necessarily a bad thing, just a prediction (in any
case, I did that too and it was fun :-)
Another data point... look at how many Classpath-based VM's use a
"stock" version of Classpath. I'd guess it's less than half. Why?
Because various VM peformance and/or design issues mean they need
to change things. If we can't even get all the existing Classpath
VM's to conform to the same Classpath-VM API, which is overall much
smaller than a modular VM API would be, that doesn't bode well.
Finally, I'm happy to offer any portions of JCVM that would be useful
to Harmony, and can change the license if necessary (I'm not religous
about licensing). E.g., some of the tedious stuff like the ZIP file
reader (based on libz).
Cheers,
-Archie
[1] http://jcvm.sourceforge.net/
__________________________________________________________________________
Archie Cobbs * CTO, Awarix * http://www.awarix.com
Re: State of the World
Posted by Ben Laurie <be...@algroup.co.uk>.
Tom Tromey wrote:
> * Fragmentation. I think there are too many free JVMs. In particular
> the "C/C++-based VM" niche is over-full. It ought to be possible,
> though difficult, to write a configurable core VM that can be reused
> by other projects. The idea here is, have a reusable reference
> implementation, and reduce writing a VM to writing an execution
> engine (and perhaps a GC).
This is certainly my understanding of the "modularity" aim, and one of
my two main interests in this project (that and security, natch).
Like they said, pointers to where you think the natural fault-lines in a
JVM are would be most illuminating.
Cheers,
Ben.
--
http://www.apache-ssl.org/ben.html http://www.thebunker.net/
"There is no limit to what a man can do or how far he can go if he
doesn't mind who gets the credit." - Robert Woodruff
Re: State of the World
Posted by "Geir Magnusson Jr." <ge...@apache.org>.
On May 9, 2005, at 12:37 PM, Brian Goetz wrote:
>> It ought to be possible,
>> though difficult, to write a configurable core VM that can be
>> reused
>> by other projects. The idea here is, have a reusable reference
>> implementation, and reduce writing a VM to writing an execution
>> engine (and perhaps a GC).
>>
>
> For the sake of making sure everyone understands these vague terms
> in the same way, could you elucidate your sense of the boundaries
> between the VM and the execution engine, and why this is a sensible
> place for an abstraction barrier?
>
And feel free to offer diagrams... We'll have a website up Real Soon
Now, and can also add a wiki for this kind of discussion
geir
>
--
Geir Magnusson Jr +1-203-665-6437
geirm@apache.org
Re: State of the World
Posted by Alejandro González <ag...@NETexplora.com>.
> So for me one of the primary goals of Harmony would be to create a
> framework inside which specialist VM's could be implemented.
The modular design of the Harmony VM will make easier the testing of
experimental GCs, class validators, etc... That is a great feature for
researchers. Imagine how easier would be the transition from a traditional
VM to a multi-tasking VM like
http://java.sun.com/developer/technicalArticles/Programming/mvm/.
Re: State of the World
Posted by Matthew French <ma...@camary.co.za>.
On Mon, 2005-05-09 at 15:37 -0400, Brian Goetz wrote:
> > It ought to be possible,
> > though difficult, to write a configurable core VM that can be reused
> > by other projects. The idea here is, have a reusable reference
> > implementation, and reduce writing a VM to writing an execution
> > engine (and perhaps a GC).
>
> For the sake of making sure everyone understands these vague terms in
> the same way, could you elucidate your sense of the boundaries between
> the VM and the execution engine, and why this is a sensible place for an
> abstraction barrier?
I'll bite, even though I did not make the original comment.
A Java VM can be considered as three separate core functions[1]:
- client interface: command line, library locater, and OS abstraction
(not to be confused with library abstraction: the OS abstraction does
small things like knowing that semicolons separate the classpath instead
of colons)
- byte code interpreter (execution engine?): this is the section
responsible for turning the byte code into program steps. This can be a
simple interpreter, a just-in-time compiler, or a brute force compiler.
- garbage collector: responsible for allocating and cleaning memory.
Also responsible for soft pointers.
I have deliberately ignored functionality like file and network handling
because I believe these are purely library functions. OS dependant, but
still library.
I have also excluded classic VM functionality like threading because I
believe this depends on the execution engine being used.
Now why would one make this distinction?
One of the the big benefits of Java is its ability to "run anywhere".
Now I don't believe operating system independence is that important,
recompiling the code works in many cases. But running in different
environments *is* important:
- running inside a webserver
- as part of a HPC cluster
- on a cell phone
- inside a debugger
- from the browser
- as a command line application
We should not underestimate the power that re-using the same code in a
very different space gives us.
However, each of these environments presents different challenges: a web
server needs aggressive performance optimisations and paranoid
threading. A cell phone would rather save the memory and so do pure
interpretation without any compiling, possibly using co-operative
threading. A debugger needs to add hooks and checks. A HPC cluster needs
to do the inter-server communication and locking, but might want to
ignore bounds checking on arrays for performance. A command line
application would need pre-compiled code and fast loading, and JIT for
uncompiled code.
To a lesser extent, the choice of garbage collector presents similar
issues. A web server could need a lazy collector that uses spare cycles
when available, or perhaps an aggressive collector if the application
tends to allocate many large memory chunks.
The client interface depends on where you are running. The execution
engine depends on what you are running, and the GC depends on how it
runs.
The question seems to me to be similar to the micro- verses monolithic
kernel debate. Sun have given us the monolithic VM. Now we need the
modular VM. Except that in this case we don't have the complexity
trade-off that micro-kernels faced.
I have not been on this list long enough to really understand the
purpose behind Harmony. But one of the big benefits of open source is
that it allows the end user to choose what they use the code for instead
of being handed a black box.
So for me one of the primary goals of Harmony would be to create a
framework inside which specialist VM's could be implemented.
Even if we only implement a simple byte code interpreter, this
flexibility will make it possible for others who have the inclination to
create VM's that are better suited for other problems.
Hopefully that begins to answer the question?
:)
- Matthew
[1] For the purposes of this discussion. And off the top of my head, I
still need to consider the problem more deeply... Ohhhhmmmmmm...
Re: State of the World
Posted by Brian Goetz <br...@quiotix.com>.
> It ought to be possible,
> though difficult, to write a configurable core VM that can be reused
> by other projects. The idea here is, have a reusable reference
> implementation, and reduce writing a VM to writing an execution
> engine (and perhaps a GC).
For the sake of making sure everyone understands these vague terms in
the same way, could you elucidate your sense of the boundaries between
the VM and the execution engine, and why this is a sensible place for an
abstraction barrier?