FYI, playing with Sling, GraalVM and native images

[Bertrand Delacretaz <bd...@apache.org>]

Hi,

FYI I'm running a series of experiments that I'll present at the
adaptTo() conference, and keeping a journal at
https://issues.apache.org/jira/browse/SLING-8556

If someone has tips & tricks that might help they are welcome!

At this point the status at SLING-8556 is that creating a native image
of an OSGi app looks quite complicated, I'm switching to using our
OSGi mocks library to wire components statically.

Initial very early experiments at
https://github.com/bdelacretaz/sling-graalvm, will move that to the
Sling whiteboard soon.

-Bertrand

Re: FYI, playing with Sling, GraalVM and native images

[Bertrand Delacretaz <bd...@apache.org>]

H Paul,

Thank you very much for your input!

On Wed, Jul 3, 2019 at 5:30 PM Paul Bjorkstrand
<pa...@gmail.com> wrote:
> ...From what I know of GraalVM/native-image, the biggest hurdles will be with
> classloading (lazy load/hiding bundle-private classes) and reflection...

Yes out of the list at [1] that's what I'm mostly seeing in my experiments.

Figuring out what's causing trouble when adding third-party libraries
can be quite hard as native-image error messages are far from obvious
to me, so I've been going in very small steps and that helps.

> ...The nature of how Felix/OSGi hides nonpublic classes (like you stated #4 on
> SLING-8556) would make it extremely difficult to get this working without
> major changes...

yes, and it looks like just having classes embedded in bundles is a
problem at build time, so I'm not sure if Felix Connect would help
much (but happy to be proven wrong on that one).

>
> Reflection can be worked around, by providing a list of what can be found
> by reflection...

Yes and the tracing agent mentioned at SLING-8556 helps with that.

> ....It is likely that the entirety of Sling would need to
> be able to run without being inside an OSGi container, before you can get
> native-image to work properly....

Agreed - for now I'm focusing on getting a useful subset of Sling to
run as a native image, using static wiring of the required components.

I have moved my code to
https://github.com/apache/sling-whiteboard/tree/master/graalvm - large
parts of it are still Very Ugly (TM), for now I've been able to run
the SlingRequestProcessor in a degraded way, with lots of shortcuts
and mocks - but I do have an HTTP server that starts in a few msec and
runs some Sling core code.

I'll work on fleshing out the SlingRequestProcessor environment next,
to make the example more realistic - but I'm not shooting to run the
whole of Sling in a native image, just some small parts as mentioned.

-Bertrand

[1] https://github.com/oracle/graal/blob/master/substratevm/LIMITATIONS.md

Re: FYI, playing with Sling, GraalVM and native images

[Paul Bjorkstrand <pa...@gmail.com>]

From what I know of GraalVM/native-image, the biggest hurdles will be with
classloading (lazy load/hiding bundle-private classes) and reflection. The
nature of how Felix/OSGi hides nonpublic classes (like you stated #4 on
SLING-8556) would make it extremely difficult to get this working without
major changes.

Reflection can be worked around, by providing a list of what can be found
by reflection, so native-image can make that statically available to the
stripped down application.

Removing reflection and dynamic class loading means it would be impossible
(or nearly so) to make running application bundle changes. At that point,
you might as well be not using OSGi at all.

The idea of PojoSR is interesting, though, and (as Robert said) would be a
good starting point. It is likely that the entirety of Sling would need to
be able to run without being inside an OSGi container, before you can get
native-image to work properly. That effort, while probably huge, may be
easier than getting Felix to work with native-image.

Paul Bjorkstrand


On Wed, Jul 3, 2019 at 6:08 AM Robert Munteanu <ro...@apache.org> wrote:

> On Wed, 2019-07-03 at 13:00 +0200, Bertrand Delacretaz wrote:
> > Hi Robert,
> >
> > On Wed, Jul 3, 2019 at 12:25 PM Robert Munteanu <ro...@apache.org>
> > wrote:
> > > ...Have you considered using FelixConnect [1]
> > > instead of the OSGi mocks?...
> >
> > Yes that's one of the options as well.
> >
> > I haven't found docs or tests for that module however so I'm a bit
> > wary of using it, but I'll keep it in the back of my mind!
>
> Oak has a module using Felix Connect - back when it was called PojoSR.
>
>   https://github.com/apache/jackrabbit-oak/tree/trunk/oak-pojosr
>
> If you decide to try it out it can be a good starting point.
>
> Thanks,
>
> Robert
>
>

Re: FYI, playing with Sling, GraalVM and native images

[Robert Munteanu <ro...@apache.org>]

On Wed, 2019-07-03 at 13:00 +0200, Bertrand Delacretaz wrote:
> Hi Robert,
> 
> On Wed, Jul 3, 2019 at 12:25 PM Robert Munteanu <ro...@apache.org>
> wrote:
> > ...Have you considered using FelixConnect [1]
> > instead of the OSGi mocks?...
> 
> Yes that's one of the options as well.
> 
> I haven't found docs or tests for that module however so I'm a bit
> wary of using it, but I'll keep it in the back of my mind!

Oak has a module using Felix Connect - back when it was called PojoSR.

  https://github.com/apache/jackrabbit-oak/tree/trunk/oak-pojosr

If you decide to try it out it can be a good starting point.

Thanks,

Robert

Re: FYI, playing with Sling, GraalVM and native images

[Bertrand Delacretaz <bd...@apache.org>]

Hi Robert,

On Wed, Jul 3, 2019 at 12:25 PM Robert Munteanu <ro...@apache.org> wrote:
> ...Have you considered using FelixConnect [1]
> instead of the OSGi mocks?...

Yes that's one of the options as well.

I haven't found docs or tests for that module however so I'm a bit
wary of using it, but I'll keep it in the back of my mind!

-Bertrand

Re: FYI, playing with Sling, GraalVM and native images

[Robert Munteanu <ro...@apache.org>]

Hi Bertrand,

On Wed, 2019-07-03 at 12:04 +0200, Bertrand Delacretaz wrote:
> Hi,
> 
> FYI I'm running a series of experiments that I'll present at the
> adaptTo() conference, and keeping a journal at
> https://issues.apache.org/jira/browse/SLING-8556
> 
> If someone has tips & tricks that might help they are welcome!
> 
> At this point the status at SLING-8556 is that creating a native
> image
> of an OSGi app looks quite complicated, I'm switching to using our
> OSGi mocks library to wire components statically.

This looks quite cool! Have you considered using FelixConnect [1]
instead of the OSGi mocks?

> 
> Initial very early experiments at
> https://github.com/bdelacretaz/sling-graalvm, will move that to the
> Sling whiteboard soon.

Thanks,

Robert

[1]: https://github.com/apache/felix/tree/trunk/connect