You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Dalibor Topic <ro...@kaffe.org> on 2005/12/06 13:28:03 UTC
[Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community]
Fresh start)
Tim Ellison wrote:
> Dalibor Topic wrote:
>
>>Technically speaking, I'm not sure if the whole licensing logjam around
>>the VM interface is not essentially a red herring. If my mental model of
>>where we want to go with respect to OSGi is correct, in essence
>>
>>a) partition the class library along some borders into bundles
>
>
> Agreed.
>
yay! :)
>>b) have a small enough core (library) to run the OSGi service to load
>>the bundles of the rest of the class library.
>
>
> There is a core set of classes required to bootstrap the OSGi framework
> code. The trick is to bring up enough classes to implement the
> framework, then retrospectively apply the framework rules to those
> classes. By the time the framework is instantiated and ready to load
> new classes it will have the core classes installed.
I've had something like that in mind, too, cool.
Is it absolutely necessary to have those core classes as a bundle, and
do the retrospective trick?
I'm trying to figure out if we can shave off complexity, so please bear
with my basic knowledge of OSGi semantics :)
>
>>I'd hope that the VM interface is sufficiently defined by saying "the absolute
>>core classes necessary to run a minimal OSGi implementation",
>
>
> IMHO the framework requires too many classes for this to double as the
> VM interface -- the number of classes that the VM knows intimately is
> much smaller, and what I have been calling the 'kernel' classes (about
> 18 or so types). The kernel classes can be written as a bundle
> (fragment) that plays in the class componentization model alongside LUNI
> (java.lang, java.util, java.net, java.io), SECURITY, etc.
>
>
OK. I see how one can, by programming carefully, knock things down to a
very small, minimal set of classes necessary. I don't think we have to
absolutely strive for the absolute minimum, though, as it may end up
making the interface more complicated and less useful in practice for a
few reasons:
a) people implementing the interface on their own VMs may find out that
it relies too closely on correct order of class loading for them to
easily bootstrap their not fully standard conforming VMs. Ahem. :)
b) Minimizing the set of bootstrap classes can result in complexity
being pushed out somewhere else, for example by mandating native methods.
Considering that we have 4000-5000 classes in 1.5 to work with, I
believe the difference between an 18 class bootstrap interface and a 50
class bootstrap interface is negligible, given that, as I explained
below, most people are either able to pick various open source
implementations of a large set of those classes under licenses ranging
from BSD, ASLv2, GPL+linking exception, to LGPL, and GPL, to name a few,
and mix and match those largely to their heart's content.
So if stuffing a class into the bootstrap set helps up reduce the VM
interface complexity, I'd say let's go for it. Examples below.
>>which would
>>translate to "OSGi/Minimum-1.1 execution requirements" being the fundamental
>>VM interface, as that's what's needed to run a minimal OSGi implementation
>>that takes care of all the rest wrt to handling bundles, as far as I can
>>guess. I hope Tim will correct me if I am wrong. The minimal 'profile'
>>is, afaict, a small subset of io, lang, lang.*, net, util and security
>>packages.
>
>
> If we squeeze the VM interface down to the few kernel classes, the only
> thing that remains in the VM interface is a regular JNI impl and some
> minor additional functions (to get VM system properties etc.).
>
> Here's the big picture:
> http://svn.apache.org/viewcvs.cgi/*checkout*/incubator/harmony/enhanced/trunk/sandbox/contribs/ibm_core/doc/vm_doc/html/index.html
>
> (see the 'VMI interface' and 'Kernel Java classes' links in that doc for
> specifics)
Cool.
Here are my examples of reducing complexity by adding classes to the
bootstrap set: I see that the interface specifies fdlibm, which I assume
is needed to properly implement double/float. Rather than specifying an
interface in C with JNI and all that fun, let's just make Double and
Float implementations part of the bootstrap set.
Same with zlib: I didn't see it being specified in OSGi/Minimal, so I
guess we may be able to get by and say that java.util.zip is either part
of the bootstrap set, and then we chuck it in, and all the native
dependencies disappear. There are various java.util.zip implementations
out there, etc, etc.
In essence, I'd like to see if we can do away with the JNI methods and
the C part of the VM interface, by enlarging the bootstrap set by those
classes that would need them, and just leaving their internals unspecified.
That would mean that the vmi.h and friends are useful for runtimes using
Harmony's classes, just as the GNU Classpath interface is useful for
runtimes using those classes, and so on. The overarching VM interface
could abstract away from those, I hope.
>
>>It would, I believe, be easiest to say, the VM interface are those
>>classes, rather than following the OSGi convention of enumerating the
>>methods along with the classes.
>
>
> Agreed. To get a clean set of classes there are some non-Java-API
> methods that form part of the kernel classes' contract with the rest of
> the class library (e.g. ThreadGroup.add() and ThreadGroup.remove()).
> Without this there would be many more types pulled into the 'kernel'.
>
> Of course, VMs could choose to implement any superset of the kernel
> classes that suit them, or implement the kernel classes in any manner
> that suits them (e.g. the earlier discussion of delegates vs. in-lines I
> think is irrelevant if we agree it is the VM's perogative).
Yup. Amazing how much simpler things become but just assuming they are
there. We don;t have to wrestle with the right way to write inlineable
code, yay :)
As I said above, I don't think we need to strive for absolute
minimalism, though. I'd trade an interface that I can implement on a
lazy weekend for an interface that saves me 200k of kernel classes
download, but takes me a month to get written. Simpler interface ->
faster, easier adoption, I hope.
>
>>With that out of the way, the real challenge would be finding nice ways
>>to partition the remaining class libraries, and as you've noted before, one
>>obvious area to go for are the xml classes. Tim posted a writeup of a
>>few more ideas, which looked pretty cool.
>>
>>On the implementation side, Harmony is already in great shape with the
>>contributions from IBM and Intel, though I do not know if they allow us
>>to run a minimal OSGi implementation yet. So another challenge would be
>>to fill in the gaps, if there are any.
>
>
> I believe there is enough there already, but if people have other
> requirements I'd be happy to discuss on the list.
>
Cool. Thanks!
>
>>Finally, we'd need to have our own, ASLv2 licensed OSGi implementation.
>>I am not sure if there is one, but I hope Geir knows more.
>
>
> We are in luck:
> http://incubator.apache.org/projects/felix.html
>
Yay!
> and there are other implementations such as
> http://www.knopflerfish.org/index.html
> and
> http://www.eclipse.org/equinox/
>
Double yay! I also found Oscar and JEFFE, so there seems to be
implementations aplenty for all licenses around.
> that make this a pretty safe and vibrant area to step into.
>
>
>>So, this would be my outline for the VM interface, as I have it in my
>>mind, 1000ft view, circumventing all licensing issues by using OSGi on
>>top of a minimal set of classes we assume as given.
>>
>>How a VM implements those classes, under which license, is not our
>>problem. Harmony will have all those classes under ASLv2, other
>>projects will have GNU Classpath's implementation under GPL+linking
>>exception, different projects may chose to use the old Kaffe or gcj
>>code, or take the BSD-licensed code from Wonka as a basis for their
>>efforts.
>
>
> (Tim breaks out into an Hallelujah Chorus :-) )
Amen, brother, and all that.
>
>>Does it sound feasible? I believe it is the easiest way to go forward
>>fast, rather than getting stuck on who owns what under which license and
>>how they may or may not interact. Harmony already has a sizeable chunk
>>of the minimal needed set of classes, and so has GNU Classpath,
>>nevermind potential contributors using proprietary implementations.
>
>
> +lots.
>
Great, we are in violent agreement! Someone make a picture! :)
cheers,
dalibor topic
p.s. you know that after talking about all this stuff for the next week,
I'll drag you come for FOSDEM to give us an introduction into how it all
fits together, and why, and so on.
Re: [Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community]
Fresh start)
Posted by "Richard S. Hall" <he...@ungoverned.org>.
Dalibor Topic wrote:
> Thanks for the explanation, Richard. One last question: are the
> bundles/packages made for R4 loadable with R2/R3, i.e. do osgi
> implementations ignore attributes they don't understand, for example?
>
Bundles for R2/R3 will run on R4. It is possible to make an R4 bundle
that will work on a R3 framework, but if the bundle was designed to use
any R4 specific features, then it is likely it will not work.
-> richard
Re: [Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community] Fresh start)
Posted by Dalibor Topic <ro...@kaffe.org>.
On Wed, Dec 07, 2005 at 01:34:22PM -0500, Richard S. Hall wrote:
> Dalibor Topic wrote:
> >In terms of using a minimal OSGi environment for partitioning and
> >management of class library parts, what differences would be relevant
> >between R2/R3/R4?
> >
>
> Between R2 and R3, not much...you can pretty much consider those two
> equivalent.
>
> R4 adds some considerable differences to prior releases that may be
> relevant for you, such as:
>
> * Package import are now allowed to specify version ranges.
> * Imports and exports can specify arbitrary attributes to help
> direct how resolving occurs in a simplistic way.
> * Support for sharing multiple versions of the same classes at the
> same time.
> * Along with the above, an improved consistency model that allows an
> exported package to declare "uses" relationships on its imports
> (and/or with other exported packages in the same bundle) so that
> the framework can make sure that a given class space doesn't see
> more than one version of a given class.
> * Other forms of dependencies and packaging, such as explicit bundle
> dependencies (as opposed to package dependencies) and bundle
> fragments (that allow a logical bundle to be delivered in multiple
> physical bundles). [These features are more advanced and can be
> abused.]
>
> These are some of the main things that I can think of off the top of my
> head. At this point, Felix supports the first four bullets.
Thanks for the explanation, Richard. One last question: are the
bundles/packages made for R4 loadable with R2/R3, i.e. do osgi
implementations ignore attributes they don't understand, for example?
cheers,
dalibor topic
>
> I will actually be giving a presentation at ApacheCon in San Diego on
> this exact topic, so if anyone is interested in more details, please
> stop by and listen. If you have any other questions, feel free to ask me.
>
> -> richard
Re: [Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community]
Fresh start)
Posted by "Richard S. Hall" <he...@ungoverned.org>.
Dalibor Topic wrote:
> In terms of using a minimal OSGi environment for partitioning and
> management of class library parts, what differences would be relevant
> between R2/R3/R4?
>
Between R2 and R3, not much...you can pretty much consider those two
equivalent.
R4 adds some considerable differences to prior releases that may be
relevant for you, such as:
* Package import are now allowed to specify version ranges.
* Imports and exports can specify arbitrary attributes to help
direct how resolving occurs in a simplistic way.
* Support for sharing multiple versions of the same classes at the
same time.
* Along with the above, an improved consistency model that allows an
exported package to declare "uses" relationships on its imports
(and/or with other exported packages in the same bundle) so that
the framework can make sure that a given class space doesn't see
more than one version of a given class.
* Other forms of dependencies and packaging, such as explicit bundle
dependencies (as opposed to package dependencies) and bundle
fragments (that allow a logical bundle to be delivered in multiple
physical bundles). [These features are more advanced and can be
abused.]
These are some of the main things that I can think of off the top of my
head. At this point, Felix supports the first four bullets.
I will actually be giving a presentation at ApacheCon in San Diego on
this exact topic, so if anyone is interested in more details, please
stop by and listen. If you have any other questions, feel free to ask me.
-> richard
Re: [Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community]
Fresh start)
Posted by Dalibor Topic <ro...@kaffe.org>.
Richard S. Hall wrote:
> Equinox is an R4 compliant framework and KF is currently R3, but I have
> heard that they have a new version for R4 in the offing.
>
Thanks, Richard.
In terms of using a minimal OSGi environment for partitioning and
management of class library parts, what differences would be relevant
between R2/R3/R4?
cheers,
dalibor topic
Re: [Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community]
Fresh start)
Posted by "Richard S. Hall" <he...@ungoverned.org>.
Dalibor Topic wrote:
> Tim Ellison wrote:
>
>> Dalibor Topic wrote
>>> Finally, we'd need to have our own, ASLv2 licensed OSGi implementation.
>>> I am not sure if there is one, but I hope Geir knows more.
>>>
>> We are in luck:
>> http://incubator.apache.org/projects/felix.html
>>
>>
>
> Yay!
>
>
>> and there are other implementations such as
>> http://www.knopflerfish.org/index.html
>> and
>> http://www.eclipse.org/equinox/
>>
>>
>
> Double yay! I also found Oscar and JEFFE, so there seems to be
> implementations aplenty for all licenses around.
FYI: The JEFFREE OSGi impl is long dead...I think it is based on OSGi
R2, but I am not certain. Oscar is essentially dormant at this point
too, although I am contemplating another minor release; Oscar is based
on OSGi R3. Development started on Oscar 2.0 which is based on OSGi R4
(the new spec), but this work has now become part of the Apache Felix
project. So, of these three OSGi impls, Felix is the way to go.
Equinox is an R4 compliant framework and KF is currently R3, but I have
heard that they have a new version for R4 in the offing.
-> richard
Re: [Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community] Fresh start)
Posted by Dalibor Topic <ro...@kaffe.org>.
On Wed, Dec 07, 2005 at 12:21:37PM -0800, Matt Benson wrote:
> --- Dalibor Topic <ro...@kaffe.org> wrote:
> [SNIP]
> > The wonderful part of that story is that noone needs
> > to share any code
> > of any component: how VMs implement the bootstrap
> > set of classes, which
> > OSGi implementation they chose, if they use JNI or
> > avian carrier
> > pidgeons :) fails to matter, and noone needs to
>
> Real quick:
> are there "non-avian" carrier pigeons? ;)
>
In practice, a lot of carrier pigeons end up being non-avian in
transmission. ;)
See http://www.cix.co.uk/~mhayhurst/jdhayhurst/pigeon/pigeon.html
for practical applications and hard data ;)
> -Matt
>
>
>
>
>
> __________________________________________
> Yahoo! DSL ? Something to write home about.
> Just $16.99/mo. or less.
> dsl.yahoo.com
>
Re: [Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community]
Fresh start)
Posted by Dalibor Topic <ro...@kaffe.org>.
Tim Ellison wrote:
> Dalibor Topic wrote:
> Agreed -- and if we agree on using OSGi bundles as the technology for
> generic class library interop then I think we simply need to agree on
> the small number of non-JSE API dependencies between the bundles to
> allow for component interop.
+lots:)
> I'm inclined to start with the higher level components and work down
> towards those that require knowledge of bootsequence etc. I believe we
> can get early agreement on the interface to components that have no VM
> dependencies (i.e. those that will be most likely to be implemented in
> pure Java code), and work from there. Even demonstrating this simple
> use case will show how we can/will interop.
Cool.
> Agreed, and some VMs will be prepared to work on both models.
Yup. I think Kaffe would provide a bad testbed for that, because of the
licensing jam (can't ship the new code easily). Archie's contribution is
ASLv2, though, and it has been able to interact with GNU classpath in
the past :) so it should work as a better test bed for evaluating and
merging both interface models.
> I think
> this gives people the right choice about where they want to put their
> effort. VMs can target one, the other, or both class libraries; class
> lib developers can contribute to one, the other, or both; and users can
> mix and match the components to get whatever characteristics are on offer.
Yeah.
>
>>The wonderful part of that story is that noone needs to share any code
>>of any component: how VMs implement the bootstrap set of classes, which
>>OSGi implementation they chose, if they use JNI or avian carrier
>>pidgeons :) fails to matter, and noone needs to listen to arguments
>>about the ideal way to bind a class library to a VM, or what the best
>>OSGi implementation is. :)
>
>
> Yes, though I'd still like to have that discussion at some point because
> that would be the grand unification. It's also the mechanism we will be
> using for VMs developed under the Hy umberella.
Sure, same here. We don't have to run the full mile at once at full
speed though. The 1.5 certed free runtime goal is huge and painful
enough, to naturally lead to unification at some point in time, when the
cost of not unifying implementations is higher than maintaining separate
ones. That point should be reached in a few years (worst case), and by
then I assume all the possible licensing bridges will exist in one form
or another, so that stuff would no longer be an obstacle.
> Just want to clarify the distinction between the OS portability layer
> and the VMI. The OS portability layer is in the Hy project already, so
> that is not your 'problem' as a VM-guy.
>
> Providing an implementation of a few non-portable functions for the VMI
> is your problem; and I promise to write down exactly what they are
> (since I agree they are not easily deduced from the VMI documentation).
Yeah, I know, I was painting it with a much too broad brush by
concentrating on the portability layer. I would like to postpone the
grand unification discussion until we've got practical experience with
different implementations in a single VM, though. I believe then we'll
have a better picture how the different interfaces can be merged, and
what the respective advantages are.
> I hope that this is another mis-communication area, so I'll take it as a
> challenge to persuade you otherwise ;-)
Let's take it to private mail. That's the best way to quickly correct
miscommunication on my side in my experience. :)
>>>The summary is :
>>>(1) we should aim for complimentary classlib componentization to get
>>>interop of large functional units;
>>
>>That's the interesting part for me.
>>
>
>
> Then let's show what can be done.
>
Good.
It would be nice to know what sort of concrete benefits one could expect
as a user / developer / runtime developer from a more modular core class
library. I have a rather vague picture about concrete pros that would
provide for Kaffe, but I assume you've given the pros and cons a lot of
thought already, so it'd be nice to see what the use cases you have in mind.
cheers,
dalibor topic
Re: [Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community]
Fresh start)
Posted by Tim Ellison <t....@gmail.com>.
Dalibor Topic wrote:
> The interface in the Wiki is going to be implemented by the Apache
> Harmony class of VMs, and there specifying the need for JNI, etc. is OK.
Agreed.
> I believe you're talking about that interface which is specific to
> Apache Harmony class of VMs and the Apache Harmony class library
> implementation.
Well I was switching between a discussion of the VM<->Hy Class lib
interface and a discussion of generic classlib interfacing with OSGi,
within the same note, just to confuse you ;-)
> Let's assume that the other VMs out there also have their own ways of
> binding to a class library, and vice versa, portability helper
> functions, and so on. The other runtimes out there may want to reuse
> Apache's code, but chances are they have their own stuff for running
> OSGi code already with the class libs they use, be they proprietary or
> open, so let's just make it as simple as possible to leverage what they
> have.
Agreed -- and if we agree on using OSGi bundles as the technology for
generic class library interop then I think we simply need to agree on
the small number of non-JSE API dependencies between the bundles to
allow for component interop.
> For the OSGi partitioning work, all people we to agree on is a and b
> from the previous mail.
>
> So when I was talking about the VM interface, I mean a core set of
> bootstrap classes. I just meant a minimal set of classes that need to be
> present to be able to work with a minimal OSGi implementation, and the
> resulting class library partitioning.
I'm inclined to start with the higher level components and work down
towards those that require knowledge of bootsequence etc. I believe we
can get early agreement on the interface to components that have no VM
dependencies (i.e. those that will be most likely to be implemented in
pure Java code), and work from there. Even demonstrating this simple
use case will show how we can/will interop.
> How a VM interfaces with its own class library, then no longer matters.
> Neither does it matter how the VM partitions that bootstrap class set
> internally: Harmony may prefer to partition the bootstrap set into
> different subsets that are more efficient given its kernel layout, other
> VMs may just chose to just lump the classes together.
Agreed, and some VMs will be prepared to work on both models. I think
this gives people the right choice about where they want to put their
effort. VMs can target one, the other, or both class libraries; class
lib developers can contribute to one, the other, or both; and users can
mix and match the components to get whatever characteristics are on offer.
> The wonderful part of that story is that noone needs to share any code
> of any component: how VMs implement the bootstrap set of classes, which
> OSGi implementation they chose, if they use JNI or avian carrier
> pidgeons :) fails to matter, and noone needs to listen to arguments
> about the ideal way to bind a class library to a VM, or what the best
> OSGi implementation is. :)
Yes, though I'd still like to have that discussion at some point because
that would be the grand unification. It's also the mechanism we will be
using for VMs developed under the Hy umberella.
> More interestingly, though, everyone can use whichever component they
> want internally, and it won't matter if it's proprietary of not, as long
> as the bootstrap set of classes is there, a minimal osgi implementation
> is included, and the partitioning is agreed upon.
Yes, let's start there since attempts to start bottom-up have floundered.
<snip>
>>Take a look at the interface and how much is already implemented for
>>you, and let me know if you think it is not a lazy weekend job. The
>>Doxygen includes all the code that is in the Hy codebase already -- I
>>guess it would be helpful to have a step-by-step guide of what a VM
>>needs to do.
>
> Oh, it's not that I don't think I could not write yet another
> portability layer into Kaffe, it's just a bit superfluous to add a third
> one, since it already has two of them: one for the core vm, and one from
> GNU classpath's class libraries.
Just want to clarify the distinction between the OS portability layer
and the VMI. The OS portability layer is in the Hy project already, so
that is not your 'problem' as a VM-guy.
Providing an implementation of a few non-portable functions for the VMI
is your problem; and I promise to write down exactly what they are
(since I agree they are not easily deduced from the VMI documentation).
> It's not that I don't like portability layers, either, it's just that
> they create a lot of boring maintainance work as soon as one has several
> on one's hands, and has a VM that has users on a few dozens of
> platforms. That in general means facing portability problems where noone
> else has gone before, and there are nicer things to do than making some
> portability layer work with gcc2 on m68k-a/ux on a sunny friday
> afternoon. I don't want to allude that your C code isn't flawless, it's
> just that I am not interested in maintaining yet another portability
> layer if I can possibly avoid it. Been there, done that, etc. :)
I hope that this is another mis-communication area, so I'll take it as a
challenge to persuade you otherwise ;-)
> Nevermind that I'd have to write my own implementation of the C code, as
> I can't ship ASLv2 licensed code inside Kaffe (GPL2).
Yes, this could open the can of worms again, which is why agreeing on
the classlib components (as you described above) is something we should
do in parallel at least.
> While maintaining
> portability layers is bad enough, I can't imagine what would drive me to
> write another one ... [1] :)
>
>>I think so -- though I've probably confused things by talking about the
>>VMI and talking about class lib componentization in a single thread.
>>
>>The summary is :
>> (1) we should aim for complimentary classlib componentization to get
>>interop of large functional units;
>
> That's the interesting part for me.
>
Then let's show what can be done.
>>>p.s. you know that after talking about all this stuff for the next week,
>>>I'll drag you come for FOSDEM to give us an introduction into how it all
>>>fits together, and why, and so on.
>>
>>You'll have to translate that paragraph for me -- I can't parse it ;-)
>
> It means, if you come to the Classpath dev room at FOSDEM, we'll have a
> few beers together, and you can explain me IBM's grand vision for the
> class libs. :)
I'll be at ApacheCon next week and will be happy to discuss with anyone
there; plans around FOSDEM are uncertain.
> cheers,
> dalibor topic
>
> [1] Well, if I was offered one million US dollars in small bills for a
> weekend of work, then we could talk about me overcoming that aversion to
> writing portability layers in C ... :)
A guess a T-shirt won't do then :-)
Regards,
Tim
--
Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.
Re: [Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community] Fresh start)
Posted by Matt Benson <gu...@yahoo.com>.
--- Dalibor Topic <ro...@kaffe.org> wrote:
[SNIP]
> The wonderful part of that story is that noone needs
> to share any code
> of any component: how VMs implement the bootstrap
> set of classes, which
> OSGi implementation they chose, if they use JNI or
> avian carrier
> pidgeons :) fails to matter, and noone needs to
Real quick:
are there "non-avian" carrier pigeons? ;)
-Matt
__________________________________________
Yahoo! DSL Something to write home about.
Just $16.99/mo. or less.
dsl.yahoo.com
Re: [Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community]
Fresh start)
Posted by Dalibor Topic <ro...@kaffe.org>.
Tim Ellison wrote:
> Dalibor Topic wrote:
>
> Just to be clear, the kernel classes are VM-dependent types that are not
> typically reusable since the VM typically will 'know' the shape of the
> class/instances. I think it is useful to minimize that set.
>
Yeah, I think we were talking past each other here. :)
The interface in the Wiki is going to be implemented by the Apache
Harmony class of VMs, and there specifying the need for JNI, etc. is OK.
I believe you're talking about that interface which is specific to
Apache Harmony class of VMs and the Apache Harmony class library
implementation.
Let's assume that the other VMs out there also have their own ways of
binding to a class library, and vice versa, portability helper
functions, and so on. The other runtimes out there may want to reuse
Apache's code, but chances are they have their own stuff for running
OSGi code already with the class libs they use, be they proprietary or
open, so let's just make it as simple as possible to leverage what they
have.
For the OSGi partitioning work, all people we to agree on is a and b
from the previous mail.
So when I was talking about the VM interface, I mean a core set of
bootstrap classes. I just meant a minimal set of classes that need to be
present to be able to work with a minimal OSGi implementation, and the
resulting class library partitioning.
How a VM interfaces with its own class library, then no longer matters.
Neither does it matter how the VM partitions that bootstrap class set
internally: Harmony may prefer to partition the bootstrap set into
different subsets that are more efficient given its kernel layout, other
VMs may just chose to just lump the classes together.
The wonderful part of that story is that noone needs to share any code
of any component: how VMs implement the bootstrap set of classes, which
OSGi implementation they chose, if they use JNI or avian carrier
pidgeons :) fails to matter, and noone needs to listen to arguments
about the ideal way to bind a class library to a VM, or what the best
OSGi implementation is. :)
More interestingly, though, everyone can use whichever component they
want internally, and it won't matter if it's proprietary of not, as long
as the bootstrap set of classes is there, a minimal osgi implementation
is included, and the partitioning is agreed upon.
> You may prove me wrong now, but are there any VMs targetting JSE that
> don't do JNI?
Yes. There are platforms where C is not the lingua franca. GNU Classpath
has a few such users, afair.
> If we raise the semantic level far enough Classpath and Harmony are
> bound to be able to interoperate at some level <g>
>
That may be desireable, yeah. I can hope. :)
> Take a look at the interface and how much is already implemented for
> you, and let me know if you think it is not a lazy weekend job. The
> Doxygen includes all the code that is in the Hy codebase already -- I
> guess it would be helpful to have a step-by-step guide of what a VM
> needs to do.
Oh, it's not that I don't think I could not write yet another
portability layer into Kaffe, it's just a bit superfluous to add a third
one, since it already has two of them: one for the core vm, and one from
GNU classpath's class libraries.
It's not that I don't like portability layers, either, it's just that
they create a lot of boring maintainance work as soon as one has several
on one's hands, and has a VM that has users on a few dozens of
platforms. That in general means facing portability problems where noone
else has gone before, and there are nicer things to do than making some
portability layer work with gcc2 on m68k-a/ux on a sunny friday
afternoon. I don't want to allude that your C code isn't flawless, it's
just that I am not interested in maintaining yet another portability
layer if I can possibly avoid it. Been there, done that, etc. :)
Nevermind that I'd have to write my own implementation of the C code, as
I can't ship ASLv2 licensed code inside Kaffe (GPL2). While maintaining
portability layers is bad enough, I can't imagine what would drive me to
write another one ... [1] :)
> I think so -- though I've probably confused things by talking about the
> VMI and talking about class lib componentization in a single thread.
>
> The summary is :
> (1) we should aim for complimentary classlib componentization to get
> interop of large functional units;
That's the interesting part for me.
>>p.s. you know that after talking about all this stuff for the next week,
>>I'll drag you come for FOSDEM to give us an introduction into how it all
>>fits together, and why, and so on.
>
>
> You'll have to translate that paragraph for me -- I can't parse it ;-)
>
It means, if you come to the Classpath dev room at FOSDEM, we'll have a
few beers together, and you can explain me IBM's grand vision for the
class libs. :)
cheers,
dalibor topic
[1] Well, if I was offered one million US dollars in small bills for a
weekend of work, then we could talk about me overcoming that aversion to
writing portability layers in C ... :)
Re: [Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community]
Fresh start)
Posted by Tim Ellison <t....@gmail.com>.
Dalibor Topic wrote:
>>There is a core set of classes required to bootstrap the OSGi framework
>>code. The trick is to bring up enough classes to implement the
>>framework, then retrospectively apply the framework rules to those
>>classes. By the time the framework is instantiated and ready to load
>>new classes it will have the core classes installed.
>
> I've had something like that in mind, too, cool.
>
> Is it absolutely necessary to have those core classes as a bundle, and
> do the retrospective trick?
No it's not. It's desirable because then you get the benefits of
componentization across the entire class library, but it is not
absolutely necessary.
If the core classes are declared in bundles then dependent bundles can
verify the version compatibility of the core classes, and in turn core
class bundles can be excluded from reaching into the non-API areas of
core, etc.
> I'm trying to figure out if we can shave off complexity, so please bear
> with my basic knowledge of OSGi semantics :)
The additional complexity may be as simple as requiring the VM to
support changing a class' class loader. When the framework is being
brought up the core classes will be loaded by the bootstrap class loader
(which is typically known by the VM). When the framework is defined and
initializing those core classes will be moved to framework-defined class
loaders (to enforce visibility rules).
There are other approaches that are not so naive, but you get the idea.
>>IMHO the framework requires too many classes for this to double as the
>>VM interface -- the number of classes that the VM knows intimately is
>>much smaller, and what I have been calling the 'kernel' classes (about
>>18 or so types). The kernel classes can be written as a bundle
>>(fragment) that plays in the class componentization model alongside LUNI
>>(java.lang, java.util, java.net, java.io), SECURITY, etc.
>
> OK. I see how one can, by programming carefully, knock things down to a
> very small, minimal set of classes necessary. I don't think we have to
> absolutely strive for the absolute minimum, though, as it may end up
> making the interface more complicated and less useful in practice for a
> few reasons:
>
> a) people implementing the interface on their own VMs may find out that
> it relies too closely on correct order of class loading for them to
> easily bootstrap their not fully standard conforming VMs. Ahem. :)
>
> b) Minimizing the set of bootstrap classes can result in complexity
> being pushed out somewhere else, for example by mandating native methods.
Just to be clear, the kernel classes are VM-dependent types that are not
typically reusable since the VM typically will 'know' the shape of the
class/instances. I think it is useful to minimize that set.
The set of classes required to run an OSGi framework is a superset of
kernel; and of course there are a number of meaningful supersets
reaching up from there. I proposed the boundaries for those as the
functional units of JSE, as defined on the wiki.
The mix-n-match approach works top-down just as well as bottom-up:
bottom-up:
If you implement a VM with the minimal VM Interface (kernel classes +
vmi.h) then you can reuse all of the remainder of the Harmony classlib code.
top-down:
If you develop an alternative implementation of some component, say
java math, then by including the packages declared in the
componentization proposal, and declaring it to be an OSGi bundle, you
can confidently provide it as a drop-in replacement to the java math
bundle in Harmony class libs.
Componentization gives you benefits at development time (e.g. explict
exports allow you to evolve the implementation with a well-managed SPI
to other bundles), and at runtime (e.g. explicit imports allow you to
co-exist with other bundles that require different versions of the same
bundle).
There is no harm at all in VMs having dependency requirements on a
particular set of bundles based on their capabilities.
> Considering that we have 4000-5000 classes in 1.5 to work with, I
> believe the difference between an 18 class bootstrap interface and a 50
> class bootstrap interface is negligible, given that, as I explained
> below, most people are either able to pick various open source
> implementations of a large set of those classes under licenses ranging
> from BSD, ASLv2, GPL+linking exception, to LGPL, and GPL, to name a few,
> and mix and match those largely to their heart's content.
Absolutely, people can bootstrap with whatever they use today, then use
components from Harmony or Classpath that are higher in the SE stack.
Mmmm, harmony.
The Java code is likely to be quite easy to split apart into components,
the native code may need massaging to be built into a separate lib to
make the component stand-alone. And if you want the component to work
on a Hy-compatible VM the only dependencies on the VM from your code
must be (you guessed it) through the JNI, or VMI, or Kernel classes.
> So if stuffing a class into the bootstrap set helps up reduce the VM
> interface complexity, I'd say let's go for it. Examples below.
<snip>
> Here are my examples of reducing complexity by adding classes to the
> bootstrap set: I see that the interface specifies fdlibm, which I assume
> is needed to properly implement double/float. Rather than specifying an
> interface in C with JNI and all that fun, let's just make Double and
> Float implementations part of the bootstrap set.
fdibm is just an artifact of the existing implementation, not part of
the VMI -- so you get Double and Float for free (i.e. VMs don't have to
make any concession to how they are implemented).
You may prove me wrong now, but are there any VMs targetting JSE that
don't do JNI? The deal is, if you can do JNI, grok the VMI(*), and you
implement the kernel classes then the remainder of the class libraries
will work.
(*) Note: the HyPortLib and HyZipCachePool defined in the Doxygen doc
are provided in the Hy classlib code. The functions the VM has to
provide are pretty simple.
> Same with zlib: I didn't see it being specified in OSGi/Minimal, so I
> guess we may be able to get by and say that java.util.zip is either part
> of the bootstrap set, and then we chuck it in, and all the native
> dependencies disappear. There are various java.util.zip implementations
> out there, etc, etc.
Same story, either you play the VMI game and reuse what is in Hy, or you
provide an alternative implementation of the ARCHIVE component (contains
JAR and Zip support).
> In essence, I'd like to see if we can do away with the JNI methods and
> the C part of the VM interface, by enlarging the bootstrap set by those
> classes that would need them, and just leaving their internals unspecified.
>
> That would mean that the vmi.h and friends are useful for runtimes using
> Harmony's classes, just as the GNU Classpath interface is useful for
> runtimes using those classes, and so on. The overarching VM interface
> could abstract away from those, I hope.
If we raise the semantic level far enough Classpath and Harmony are
bound to be able to interoperate at some level <g>
<snip>
>>Of course, VMs could choose to implement any superset of the kernel
>>classes that suit them, or implement the kernel classes in any manner
>>that suits them (e.g. the earlier discussion of delegates vs. in-lines I
>>think is irrelevant if we agree it is the VM's perogative).
>
>
> Yup. Amazing how much simpler things become but just assuming they are
> there. We don;t have to wrestle with the right way to write inlineable
> code, yay :)
>
> As I said above, I don't think we need to strive for absolute
> minimalism, though. I'd trade an interface that I can implement on a
> lazy weekend for an interface that saves me 200k of kernel classes
> download, but takes me a month to get written. Simpler interface ->
> faster, easier adoption, I hope.
Take a look at the interface and how much is already implemented for
you, and let me know if you think it is not a lazy weekend job. The
Doxygen includes all the code that is in the Hy codebase already -- I
guess it would be helpful to have a step-by-step guide of what a VM
needs to do.
<snip>
> Great, we are in violent agreement! Someone make a picture! :)
I think so -- though I've probably confused things by talking about the
VMI and talking about class lib componentization in a single thread.
The summary is :
(1) we should aim for complimentary classlib componentization to get
interop of large functional units;
(2) VMs wanting to run all the Hy classlib code can do so provided they
have a lazy weekend to spare.
> p.s. you know that after talking about all this stuff for the next week,
> I'll drag you come for FOSDEM to give us an introduction into how it all
> fits together, and why, and so on.
You'll have to translate that paragraph for me -- I can't parse it ;-)
Regards,
Tim
--
Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.
Re: [Technical] VM Interface/OSGi discussion (Was: Re: [Licensing/Community] Fresh start)
Posted by Davanum Srinivas <da...@gmail.com>.
I love it... +1 from me!!!!!! Let's please draw up a concrete plan and
wiki-it :)
-- dims
On 12/6/05, Dalibor Topic <ro...@kaffe.org> wrote:
> Tim Ellison wrote:
> > Dalibor Topic wrote:
> >
> >>Technically speaking, I'm not sure if the whole licensing logjam around
> >>the VM interface is not essentially a red herring. If my mental model of
> >>where we want to go with respect to OSGi is correct, in essence
> >>
> >>a) partition the class library along some borders into bundles
> >
> >
> > Agreed.
> >
>
> yay! :)
>
> >>b) have a small enough core (library) to run the OSGi service to load
> >>the bundles of the rest of the class library.
> >
> >
> > There is a core set of classes required to bootstrap the OSGi framework
> > code. The trick is to bring up enough classes to implement the
> > framework, then retrospectively apply the framework rules to those
> > classes. By the time the framework is instantiated and ready to load
> > new classes it will have the core classes installed.
>
> I've had something like that in mind, too, cool.
>
> Is it absolutely necessary to have those core classes as a bundle, and
> do the retrospective trick?
>
> I'm trying to figure out if we can shave off complexity, so please bear
> with my basic knowledge of OSGi semantics :)
>
> >
> >>I'd hope that the VM interface is sufficiently defined by saying "the absolute
> >>core classes necessary to run a minimal OSGi implementation",
> >
> >
> > IMHO the framework requires too many classes for this to double as the
> > VM interface -- the number of classes that the VM knows intimately is
> > much smaller, and what I have been calling the 'kernel' classes (about
> > 18 or so types). The kernel classes can be written as a bundle
> > (fragment) that plays in the class componentization model alongside LUNI
> > (java.lang, java.util, java.net, java.io), SECURITY, etc.
> >
> >
>
> OK. I see how one can, by programming carefully, knock things down to a
> very small, minimal set of classes necessary. I don't think we have to
> absolutely strive for the absolute minimum, though, as it may end up
> making the interface more complicated and less useful in practice for a
> few reasons:
>
> a) people implementing the interface on their own VMs may find out that
> it relies too closely on correct order of class loading for them to
> easily bootstrap their not fully standard conforming VMs. Ahem. :)
>
> b) Minimizing the set of bootstrap classes can result in complexity
> being pushed out somewhere else, for example by mandating native methods.
>
> Considering that we have 4000-5000 classes in 1.5 to work with, I
> believe the difference between an 18 class bootstrap interface and a 50
> class bootstrap interface is negligible, given that, as I explained
> below, most people are either able to pick various open source
> implementations of a large set of those classes under licenses ranging
> from BSD, ASLv2, GPL+linking exception, to LGPL, and GPL, to name a few,
> and mix and match those largely to their heart's content.
>
> So if stuffing a class into the bootstrap set helps up reduce the VM
> interface complexity, I'd say let's go for it. Examples below.
>
> >>which would
> >>translate to "OSGi/Minimum-1.1 execution requirements" being the fundamental
> >>VM interface, as that's what's needed to run a minimal OSGi implementation
> >>that takes care of all the rest wrt to handling bundles, as far as I can
> >>guess. I hope Tim will correct me if I am wrong. The minimal 'profile'
> >>is, afaict, a small subset of io, lang, lang.*, net, util and security
> >>packages.
> >
> >
> > If we squeeze the VM interface down to the few kernel classes, the only
> > thing that remains in the VM interface is a regular JNI impl and some
> > minor additional functions (to get VM system properties etc.).
> >
> > Here's the big picture:
> > http://svn.apache.org/viewcvs.cgi/*checkout*/incubator/harmony/enhanced/trunk/sandbox/contribs/ibm_core/doc/vm_doc/html/index.html
> >
> > (see the 'VMI interface' and 'Kernel Java classes' links in that doc for
> > specifics)
>
> Cool.
>
> Here are my examples of reducing complexity by adding classes to the
> bootstrap set: I see that the interface specifies fdlibm, which I assume
> is needed to properly implement double/float. Rather than specifying an
> interface in C with JNI and all that fun, let's just make Double and
> Float implementations part of the bootstrap set.
>
> Same with zlib: I didn't see it being specified in OSGi/Minimal, so I
> guess we may be able to get by and say that java.util.zip is either part
> of the bootstrap set, and then we chuck it in, and all the native
> dependencies disappear. There are various java.util.zip implementations
> out there, etc, etc.
>
> In essence, I'd like to see if we can do away with the JNI methods and
> the C part of the VM interface, by enlarging the bootstrap set by those
> classes that would need them, and just leaving their internals unspecified.
>
> That would mean that the vmi.h and friends are useful for runtimes using
> Harmony's classes, just as the GNU Classpath interface is useful for
> runtimes using those classes, and so on. The overarching VM interface
> could abstract away from those, I hope.
>
> >
> >>It would, I believe, be easiest to say, the VM interface are those
> >>classes, rather than following the OSGi convention of enumerating the
> >>methods along with the classes.
> >
> >
> > Agreed. To get a clean set of classes there are some non-Java-API
> > methods that form part of the kernel classes' contract with the rest of
> > the class library (e.g. ThreadGroup.add() and ThreadGroup.remove()).
> > Without this there would be many more types pulled into the 'kernel'.
> >
> > Of course, VMs could choose to implement any superset of the kernel
> > classes that suit them, or implement the kernel classes in any manner
> > that suits them (e.g. the earlier discussion of delegates vs. in-lines I
> > think is irrelevant if we agree it is the VM's perogative).
>
> Yup. Amazing how much simpler things become but just assuming they are
> there. We don;t have to wrestle with the right way to write inlineable
> code, yay :)
>
> As I said above, I don't think we need to strive for absolute
> minimalism, though. I'd trade an interface that I can implement on a
> lazy weekend for an interface that saves me 200k of kernel classes
> download, but takes me a month to get written. Simpler interface ->
> faster, easier adoption, I hope.
>
> >
> >>With that out of the way, the real challenge would be finding nice ways
> >>to partition the remaining class libraries, and as you've noted before, one
> >>obvious area to go for are the xml classes. Tim posted a writeup of a
> >>few more ideas, which looked pretty cool.
> >>
> >>On the implementation side, Harmony is already in great shape with the
> >>contributions from IBM and Intel, though I do not know if they allow us
> >>to run a minimal OSGi implementation yet. So another challenge would be
> >>to fill in the gaps, if there are any.
> >
> >
> > I believe there is enough there already, but if people have other
> > requirements I'd be happy to discuss on the list.
> >
>
> Cool. Thanks!
>
> >
> >>Finally, we'd need to have our own, ASLv2 licensed OSGi implementation.
> >>I am not sure if there is one, but I hope Geir knows more.
> >
> >
> > We are in luck:
> > http://incubator.apache.org/projects/felix.html
> >
>
> Yay!
>
> > and there are other implementations such as
> > http://www.knopflerfish.org/index.html
> > and
> > http://www.eclipse.org/equinox/
> >
>
> Double yay! I also found Oscar and JEFFE, so there seems to be
> implementations aplenty for all licenses around.
>
> > that make this a pretty safe and vibrant area to step into.
> >
> >
> >>So, this would be my outline for the VM interface, as I have it in my
> >>mind, 1000ft view, circumventing all licensing issues by using OSGi on
> >>top of a minimal set of classes we assume as given.
> >>
> >>How a VM implements those classes, under which license, is not our
> >>problem. Harmony will have all those classes under ASLv2, other
> >>projects will have GNU Classpath's implementation under GPL+linking
> >>exception, different projects may chose to use the old Kaffe or gcj
> >>code, or take the BSD-licensed code from Wonka as a basis for their
> >>efforts.
> >
> >
> > (Tim breaks out into an Hallelujah Chorus :-) )
>
> Amen, brother, and all that.
>
> >
> >>Does it sound feasible? I believe it is the easiest way to go forward
> >>fast, rather than getting stuck on who owns what under which license and
> >>how they may or may not interact. Harmony already has a sizeable chunk
> >>of the minimal needed set of classes, and so has GNU Classpath,
> >>nevermind potential contributors using proprietary implementations.
> >
> >
> > +lots.
> >
>
> Great, we are in violent agreement! Someone make a picture! :)
>
> cheers,
> dalibor topic
>
> p.s. you know that after talking about all this stuff for the next week,
> I'll drag you come for FOSDEM to give us an introduction into how it all
> fits together, and why, and so on.
>
--
Davanum Srinivas : http://wso2.com/blogs/