Pivot and OSGi

[Greg Brown <gk...@mac.com>]

Hi all,

Now that we're pretty much done with version 1.1, we're starting to think about features for the next release of Pivot, version 1.2. One topic that has been discussed is OSGi support. We understand OSGi from a high level and think that support for OSGi-style modules or bundles would be a valuable addition to Pivot. However, we don't know enough about it to know if this is something that we'd need to build from scratch or if there are existing Java libraries that we could take advantage of.

If anyone has relevant experience and would care to share your insight, we would appreciate it.

Thanks,
Greg

Re: Pivot and OSGi

[Greg Brown <gk...@mac.com>]

>The main problem with OSGi is 'blindless classloading'...
>...if you have any loadClass, you should break out and allow 
>a custom classloading hook.

I just checked, and we're not currently calling loadClass() anywhere.

>The other thing is that Pivot's artifacts being proper bundles is also
>a nice thing for OSGi developers. I recommend that the Maven Bundle
>Plugin (hosted at Felix) is used. In essence, it will assist you in
>making sure that the Bundle Manifest is correct, and remind you when
>you add new packages and forget to add them to the 'recipe'. It does
>so by introspecting the class files exclusively.

OK, we'll look into this.

Re: Pivot and OSGi

[Niclas Hedhman <ni...@hedhman.org>]

On Wed, Mar 25, 2009 at 11:33 PM, Greg Brown <gk...@mac.com> wrote:
> Hi all,
>
> Now that we're pretty much done with version 1.1, we're starting to think about features for the next release of Pivot, version 1.2. One topic that has been discussed is OSGi support. We understand OSGi from a high level and think that support for OSGi-style modules or bundles would be a valuable addition to Pivot. However, we don't know enough about it to know if this is something that we'd need to build from scratch or if there are existing Java libraries that we could take advantage of.
>
> If anyone has relevant experience and would care to share your insight, we would appreciate it.


This thread has generated more content than I have time to read now.

I have worked extensively on OSGi over the last 4-5 years, and seen
all the good and bad sides of it.

IMHO, Pivot should probably not be dependent on OSGi, and instead just
make sure that OSGi issues are taken care of (if possible).

The main problem with OSGi is 'blindless classloading', i.e. all
applications that does a loadClass() or forName() will have issues.
This includes serialization, persistence and configuration stuff. I
have not gone over the code to see how much of this Pivot do, but if
you have any loadClass, you should break out and allow a custom
classloading hook.
The other thing is that Pivot's artifacts being proper bundles is also
a nice thing for OSGi developers. I recommend that the Maven Bundle
Plugin (hosted at Felix) is used. In essence, it will assist you in
making sure that the Bundle Manifest is correct, and remind you when
you add new packages and forget to add them to the 'recipe'. It does
so by introspecting the class files exclusively.

For fooling around and testing OSGi in general, I must recommend Pax
Runner (wiki.ops4j.org), since it literally does all the setup of the
framework for you (in any of the 3 frameworks you choose), and you
should be up and running in minutes instead of hours and days. The Pax
project also have tooling for in-container testing, development aids
and others. Also, the Pax team develops PeaBerry, which is OSGi
integration for Guice, but that one is hosted on Google Code.



Cheers
Niclas

-- 
http://www.qi4j.org - New Energy for Java

Re: Pivot and OSGi

[Christopher Brind <br...@brindy.org.uk>]

Hi,

Some comments in line - will respond to Todd a little later...


2009/3/25 Sandro Martini <sa...@gmail.com>

> Hi to all,
> I've a very little experience with OSGi, but i think that a primary
> Pivot use case for this could be to handle a load/start/stop/reload


In OSGi this is install/start/stop/update/uninstall.


> module inside an application, so maybe a more simple approach could be
> enough, to avoid too much heavy Pivot applications by library
> dependencies and complex code for plugins/modules creation.



> Maybe for this some classloader tricks could be enough ... and maybe
> this simpler approach could be done as much as OSGi-compatible as we
> can.



> Java 7 will have these features (modules), so this could be a risk to
> implement by custom code (complexity and less features).
>

If you're following the JSR 294 discussion (as I am) you'll see that there
are people who are striving to make module support in Java 7 compatible with
the existing module implementations (namely OSGi and Jigsaw), so choosing
OSGi won't be a problem.


>
> > Although I think that OSGi is great technology, I don't see that it's
> > something that pivot needs to implement, other than perhaps making it
> easier
> > for OTHER people to use pivot in an OSGi based application.
> I think the same.

There are some libraries that takes a jar and wraps inside an OSGi
> bundle, someone has tried ?


Eclipse has a wizard that can take Jar files and turn them in to OSGi
bundles.  All it really does is package them in to a single jar, and export
a version controlled list of packages.  OSGi allows the same package to
exported with different versions and allows other bundles to choose the
version of the package they want to import.  Thus, jar hell is eliminated
because you can be as explicit as you want about the exact version of a
library you want to use.


>
>
> >> First, I've been thinking that we might want to start thinking about
> >> the concept of a "Pivot kernel" that can show the loading state of an
> >> app while the JARs are loading, to eliminate the unresponsive feel of
> >> the app while it initializes.
> Ok, but i think this is complex, so why not for a more future (2.0) release
> ?
> Or, for the next try with a simpler approach, then see if/what to do
> in this direction ...
>
> > I'm not sure if it's still a problem, but I tried once to implement an
> > applet loading display using a custom classloader and ran into some bugs
> in
> > the classloader/sandbox interaction in the java plugin (this was java
> 1.5).
> Right, so we should think for a future release (2.0 or maybe 1.x) to
> drop the pre-Java 6 (Update 10) compatibility, otherwise we risk to
> miss some recently fixed Java features ...
>
> Best regards,
> Sandro
>

Re: Pivot and OSGi

[Sandro Martini <sa...@gmail.com>]

Hi to all,
I've a very little experience with OSGi, but i think that a primary
Pivot use case for this could be to handle a load/start/stop/reload
module inside an application, so maybe a more simple approach could be
enough, to avoid too much heavy Pivot applications by library
dependencies and complex code for plugins/modules creation.
Maybe for this some classloader tricks could be enough ... and maybe
this simpler approach could be done as much as OSGi-compatible as we
can.
Java 7 will have these features (modules), so this could be a risk to
implement by custom code (complexity and less features).

> Although I think that OSGi is great technology, I don't see that it's
> something that pivot needs to implement, other than perhaps making it easier
> for OTHER people to use pivot in an OSGi based application.
I think the same.
There are some libraries that takes a jar and wraps inside an OSGi
bundle, someone has tried ?

>> First, I've been thinking that we might want to start thinking about
>> the concept of a "Pivot kernel" that can show the loading state of an
>> app while the JARs are loading, to eliminate the unresponsive feel of
>> the app while it initializes.
Ok, but i think this is complex, so why not for a more future (2.0) release ?
Or, for the next try with a simpler approach, then see if/what to do
in this direction ...

> I'm not sure if it's still a problem, but I tried once to implement an
> applet loading display using a custom classloader and ran into some bugs in
> the classloader/sandbox interaction in the java plugin (this was java 1.5).
Right, so we should think for a future release (2.0 or maybe 1.x) to
drop the pre-Java 6 (Update 10) compatibility, otherwise we risk to
miss some recently fixed Java features ...

Best regards,
Sandro

Re: Pivot and OSGi

[Noel Grandin <no...@gmail.com>]

Hi
Although I think that OSGi is great technology, I don't see that it's
something that pivot needs to implement, other than perhaps making it easier
for OTHER people to use pivot in an OSGi based application.


On Wed, Mar 25, 2009 at 18:50, Todd Volkert <tv...@gmail.com> wrote:

> First, I've been thinking that we might want to start thinking about
> the concept of a "Pivot kernel" that can show the loading state of an
> app while the JARs are loading, to eliminate the unresponsive feel of
> the app while it initializes.  Second, we could fall back on the


The latest java plugin displays a kind of loading thingy, and it also allows
you to specify a splash image to display while the jars are loading.

I'm not sure if it's still a problem, but I tried once to implement an
applet loading display using a custom classloader and ran into some bugs in
the classloader/sandbox interaction in the java plugin (this was java 1.5).

Regards, Noel Grandin.

Re: Pivot and OSGi

[Christopher Brind <br...@brindy.org.uk>]

Comments in line.  Have read and will respond to Greg's comments too.

2009/3/25 Todd Volkert <tv...@gmail.com>

> Wow, thanks Chris; this is an great entry point!  My
> comments/questions are inline.
>
> > I work with OSGi all the time so this sounds very appealing to me.
>
> Once this is scoped, if you're willing to tackle this, that would be
> awesome.
>

Sure. =)



>
> > But I strongly recommend that people play around with these as well:
> > Equinox (the implementation Eclipse is built on) :
> > http://www.eclipse.org/equinox/
> > Knoplerfish: http://www.knopflerfish.org/
>
> This seems like a daunting task given my lack of OSGi experience.  Do
> you think you'd be able to easily discern the relative strengths and
> weaknesses of these three projects?


Once you get your head around OSGi the questions you'll be asking are:
- right, where do I put my bundles? (though technically they can be anywhere
on the file system)
- how do I manage my bundles? (i.e. install/start/stop/update/uninstall)  In
Equinox (and Felix I think) you can issue commands at a command prompt
provided at the console and Knoplerfish provides a UI.

Then you get in to things like where the various frameworks store data for
bundles, do they unpack the jars somewhere, things like that


>
>
> > IMHO, The
> > Extensions and Extension Points approach is more suitable to building UIs
> > based applications than using the service registry and this is exactly
> how
> > the majority of the Eclipse IDE hangs together.
>
> Agreed.  I built an extension and extension point API in a JavaScript
> library (around a "fake" OSGi layer), and it's a great way to
> structure applications.


It sure is an interesting model.

However, must remember to be careful with the use of "OSGi" as it is a
trademark of the OSGi Alliance and they do protect that trademark.
Basically, If we're not using actually using the OSGi spec, we can't mention
it.


>
>
> > One problem that immediately springs to mind is that one of the core
> things
> > that OSGi provides is a data area, somewhere that bundles can read and
> write
> > files.  This maybe a problem for Pivot since it will often be running in
> a
> > sandbox with strict security policies.  As such, it maybe that we need to
> > modify whichever framework we base our work on in some way, but then we
> > potentially don't have an OSGi compliant container.  I'll have to re-read
> > the spec to work out what we can get away with.
>
> Might we be able to use the preferences API, or is that too limiting?
> We could also say that if you choose to use Pivot OSGi support, you
> must either run in a signed applet or a JNLP configuration with the
> <all-permissions/> security model.  Larger applications necessitating
> modular design will probably want native read/write access anyway...
>

There's no reason why we couldn't create a bundle that implements the
Preferences API but in the background delegates to another mechanism (the
Preferences stuff that Greg mentions in his mail, for instance).

I'm certainly excited about the prospect of OSGi enabling Pivot.  It
certainly works for other UI frameworks (namely Eclipse).

Cheers,
Chris

Re: Pivot and OSGi

[Todd Volkert <tv...@gmail.com>]

Wow, thanks Chris; this is an great entry point!  My
comments/questions are inline.

> I work with OSGi all the time so this sounds very appealing to me.

Once this is scoped, if you're willing to tackle this, that would be awesome.

> In order to use OSGi, we would have to find or build an implementation.

I strongly vote "find", unless there's a real compelling reason that
we have to build our own.  The only reason I could think of is if we'd
want a trimmed down version that suits our needs only and has a very
small footprint.

> There are quite a few implementations out there already, so we would need a
> very strong reason to re-implement it from the ground up.  The obvious first
> port of call for this project is Apache Felix since it is an Apache project:
> http://felix.apache.org/

I like the idea of exploiting the synergy created by us both being
under the Apache umbrella.

> But I strongly recommend that people play around with these as well:
> Equinox (the implementation Eclipse is built on) :
> http://www.eclipse.org/equinox/
> Knoplerfish: http://www.knopflerfish.org/

This seems like a daunting task given my lack of OSGi experience.  Do
you think you'd be able to easily discern the relative strengths and
weaknesses of these three projects?

> IMHO, The
> Extensions and Extension Points approach is more suitable to building UIs
> based applications than using the service registry and this is exactly how
> the majority of the Eclipse IDE hangs together.

Agreed.  I built an extension and extension point API in a JavaScript
library (around a "fake" OSGi layer), and it's a great way to
structure applications.

> One problem that immediately springs to mind is that one of the core things
> that OSGi provides is a data area, somewhere that bundles can read and write
> files.  This maybe a problem for Pivot since it will often be running in a
> sandbox with strict security policies.  As such, it maybe that we need to
> modify whichever framework we base our work on in some way, but then we
> potentially don't have an OSGi compliant container.  I'll have to re-read
> the spec to work out what we can get away with.

Might we be able to use the preferences API, or is that too limiting?
We could also say that if you choose to use Pivot OSGi support, you
must either run in a signed applet or a JNLP configuration with the
<all-permissions/> security model.  Larger applications necessitating
modular design will probably want native read/write access anyway...

> Also, while OSGi is quite lightweight for building command line, desktop or
> server apps (e.g. the minimum requirement bundle for Equinox ways in at
> 994k) to me that still feels a little heavy for the starting position for an
> applet based application.  Then again, perhaps Felix or Knoplerfish have
> smaller footprints, that is very possible.

Good point, though two things come to mind that might mitigate this.
First, I've been thinking that we might want to start thinking about
the concept of a "Pivot kernel" that can show the loading state of an
app while the JARs are loading, to eliminate the unresponsive feel of
the app while it initializes.  Second, we could fall back on the
"recommended only for Web Start deployment" answer, in which case the
large file download feels more natural since you're launching a
desktop app...

Re: Pivot and OSGi

[Christopher Brind <br...@brindy.org.uk>]

Hi,

I work with OSGi all the time so this sounds very appealing to me.

Firstly, here's a primer that I wrote - purely conceptual for those who know
Java and have heard OSGi, but don't really know what it is:
http://java.dzone.com/articles/osgi-primer

So, OSGi is a specification for modular and dynamic Java.  There is a core
specification, which is the minimum requirement for any framework and then
there is the service compendium which includes a whole load of useful
services that are optionally implemented by an implementation.  There are a
number of things in the service compendium that probably aren't appropriate,
e.g. HttpService.

All the specifications can be found here:
http://www.osgi.org/Specifications/HomePage

Pivot doesn't need to use/provide any of the optional services and of course
we can create our own services that application developers can then use
themselves.

In order to use OSGi, we would have to find or build an implementation.
There are quite a few implementations out there already, so we would need a
very strong reason to re-implement it from the ground up.  The obvious first
port of call for this project is Apache Felix since it is an Apache project:
http://felix.apache.org/

But I strongly recommend that people play around with these as well:
Equinox (the implementation Eclipse is built on) :
http://www.eclipse.org/equinox/
Knoplerfish: http://www.knopflerfish.org/

I know that Equinox has implementations of all the optional services, but I
couldn't speak for Felix or Knoplerfish.

There are probably others, but these are the three 'leaders' as far as I am
concerned.

The pure OSGi approach is the service registry in which services are
registered and consumed by the bundle Activators.  The whiteboard model
recommends that bundles register services for consumption rather than trying
to lookup services, but it really depends on what you're trying to do, of
course.

Eclipse has taken the core OSGi concept and, perfectly legitimately, added
an extension point registry.  An Eclipse plugin usually has a plugin.xml in
it and this declares extension points and defines which extensions are
provided by the plugin.  The registry watches bundles being installed and
reads the plugin.xml from the bundle if it is available.  IMHO, The
Extensions and Extension Points approach is more suitable to building UIs
based applications than using the service registry and this is exactly how
the majority of the Eclipse IDE hangs together.

One problem that immediately springs to mind is that one of the core things
that OSGi provides is a data area, somewhere that bundles can read and write
files.  This maybe a problem for Pivot since it will often be running in a
sandbox with strict security policies.  As such, it maybe that we need to
modify whichever framework we base our work on in some way, but then we
potentially don't have an OSGi compliant container.  I'll have to re-read
the spec to work out what we can get away with.

Also, while OSGi is quite lightweight for building command line, desktop or
server apps (e.g. the minimum requirement bundle for Equinox ways in at
994k) to me that still feels a little heavy for the starting position for an
applet based application.  Then again, perhaps Felix or Knoplerfish have
smaller footprints, that is very possible.

Hope that helps.

Please feel free to shout out questions.

Cheers,
Chris





2009/3/25 Greg Brown <gk...@mac.com>

> Hi all,
>
> Now that we're pretty much done with version 1.1, we're starting to think
> about features for the next release of Pivot, version 1.2. One topic that
> has been discussed is OSGi support. We understand OSGi from a high level and
> think that support for OSGi-style modules or bundles would be a valuable
> addition to Pivot. However, we don't know enough about it to know if this is
> something that we'd need to build from scratch or if there are existing Java
> libraries that we could take advantage of.
>
> If anyone has relevant experience and would care to share your insight, we
> would appreciate it.
>
> Thanks,
> Greg
>
>
>