Odd implementation detail of native libs and bundle cache

[Holger Hoffstätte <ho...@googlemail.com>]

Hi,

I just discovered a very odd and - at least for my use case very unhelpful
- change in behaviour of felix from 1.8 to 2.0 concerning native libraries.

You might know about the difficulties of managing libraries with
dependencies. For my project I had pursued the approach of handling Linux
platforms by temporarily copying dependencies to /tmp, loading them by
making the originally loaded JNI library have an RPATH of /tmp, and then
clean up. This is ugly, error-prone, insecure (symlink attack) etc. but
*worked* across all runtimes.

In order to improve this rather ugly hack I now tried to use an RPATH of
"$ORIGIN". This special value will make shared libaries look in the
location of the originating dependency, which is _exactly_ what I want for
bundles. And it works great! There is a slight catch that not all runtimes
unpack bundle libraries eagerly, so the libs have to be referenced in
order to show up in the bundle cache. Works fine too...except on felix 2.0.

For some reason a bundle's libraries are unpacked with any path given in
the NativeCode header (correct and good) but _into another subdirectory_.
This means that three libraries platform/a.so, platform/b.so,
platform/c.so will not end up in a common directory platform/ but rather
in directories platform/0/, platform/1/, platform/2/ and so on - which
totally breaks the great $ORIGIN hack. :(

Is there a good reason why felix would need to do this? Older versions did
not, and neither does equinox (at least not 3.5). I fully realize that the
layout of the bundle cache is "hands off" and an implementation detail,
but at this level I _must_ have some deterministic way of interacting with
the platform. IMHO it is wrong that resources or libraries with common
path prefixes do not actually end up in a common directory - not
technically, but in the "principle of least surprise" way.

Ideas or comments? I can help fix this if necessary.

thanks
Holger

Re: Odd implementation detail of native libs and bundle cache

["Richard S. Hall" <he...@ungoverned.org>]

On 10/10/09 14:46, Holger Hoffstätte wrote:
> (catching up on stuff, sorry for the late reply!)
>
> Richard S. Hall wrote:
>    
>> These changes came about because of support for native libraries in
>> fragments. Since fragments can attach to multiple hosts in R4.2, the
>> framework needs to be able to extract multiple copies, one for each host.
>>
>> The debate was where/how to do this. We could extract into the host
>> cache directory, which would make it easier to do what you want (i.e.,
>> all in one directory), but it would complicate cleaning up after
>> fragments since once it was uninstalled you'd need to remove it from all
>> current and past host cache directories.
>>
>> Instead, I decided to extract it into the fragment's cache directory
>> because when the fragment is uninstalled, everything is cleaned up in
>> one swoop. Of course, doing so requires the framework to extract into
>> separate locations to avoid collisions.
>>      
> Richard,
>
> Thank you for the explanation and already filing FELIX-1731 (I was just
> about to do that). I suspected that there was a "good" reason for the
> subdirectories, I just didn't know what it could be.
> For now I have everything working again via a runtime-specific check, i.e.
> I first try for a flat bundle cache layout and if that does not work, we
> do the old copying trick.
>
>    

You may have also noticed that I committed a patch for FELIX-1731, so 
please try again with 2.0.1 and see if that makes things better for you.

In the end, though, this is definitely an implementation detail and in 
the future if needs warrant, it might break again.

-> richard

> thanks
> Holger
>    

Re: Odd implementation detail of native libs and bundle cache

[Holger Hoffstätte <ho...@googlemail.com>]

(catching up on stuff, sorry for the late reply!)

Richard S. Hall wrote:
> These changes came about because of support for native libraries in 
> fragments. Since fragments can attach to multiple hosts in R4.2, the 
> framework needs to be able to extract multiple copies, one for each host.
> 
> The debate was where/how to do this. We could extract into the host 
> cache directory, which would make it easier to do what you want (i.e., 
> all in one directory), but it would complicate cleaning up after 
> fragments since once it was uninstalled you'd need to remove it from all 
> current and past host cache directories.
> 
> Instead, I decided to extract it into the fragment's cache directory 
> because when the fragment is uninstalled, everything is cleaned up in 
> one swoop. Of course, doing so requires the framework to extract into 
> separate locations to avoid collisions.

Richard,

Thank you for the explanation and already filing FELIX-1731 (I was just
about to do that). I suspected that there was a "good" reason for the
subdirectories, I just didn't know what it could be.
For now I have everything working again via a runtime-specific check, i.e.
I first try for a flat bundle cache layout and if that does not work, we
do the old copying trick.

thanks
Holger

Re: Odd implementation detail of native libs and bundle cache

["Richard S. Hall" <he...@ungoverned.org>]

These changes came about because of support for native libraries in 
fragments. Since fragments can attach to multiple hosts in R4.2, the 
framework needs to be able to extract multiple copies, one for each host.

The debate was where/how to do this. We could extract into the host 
cache directory, which would make it easier to do what you want (i.e., 
all in one directory), but it would complicate cleaning up after 
fragments since once it was uninstalled you'd need to remove it from all 
current and past host cache directories.

Instead, I decided to extract it into the fragment's cache directory 
because when the fragment is uninstalled, everything is cleaned up in 
one swoop. Of course, doing so requires the framework to extract into 
separate locations to avoid collisions.

Currently, I don't remember the exact details of why we extract the way 
we do. If you want to open an issue on this, I will look into it. No 
promises about finding a solution, though.

-> richard


On 10/6/09 16:08, Holger Hoffstätte wrote:
> Hi,
>
> I just discovered a very odd and - at least for my use case very unhelpful
> - change in behaviour of felix from 1.8 to 2.0 concerning native libraries.
>
> You might know about the difficulties of managing libraries with
> dependencies. For my project I had pursued the approach of handling Linux
> platforms by temporarily copying dependencies to /tmp, loading them by
> making the originally loaded JNI library have an RPATH of /tmp, and then
> clean up. This is ugly, error-prone, insecure (symlink attack) etc. but
> *worked* across all runtimes.
>
> In order to improve this rather ugly hack I now tried to use an RPATH of
> "$ORIGIN". This special value will make shared libaries look in the
> location of the originating dependency, which is _exactly_ what I want for
> bundles. And it works great! There is a slight catch that not all runtimes
> unpack bundle libraries eagerly, so the libs have to be referenced in
> order to show up in the bundle cache. Works fine too...except on felix 2.0.
>
> For some reason a bundle's libraries are unpacked with any path given in
> the NativeCode header (correct and good) but _into another subdirectory_.
> This means that three libraries platform/a.so, platform/b.so,
> platform/c.so will not end up in a common directory platform/ but rather
> in directories platform/0/, platform/1/, platform/2/ and so on - which
> totally breaks the great $ORIGIN hack. :(
>
> Is there a good reason why felix would need to do this? Older versions did
> not, and neither does equinox (at least not 3.5). I fully realize that the
> layout of the bundle cache is "hands off" and an implementation detail,
> but at this level I _must_ have some deterministic way of interacting with
> the platform. IMHO it is wrong that resources or libraries with common
> path prefixes do not actually end up in a common directory - not
> technically, but in the "principle of least surprise" way.
>
> Ideas or comments? I can help fix this if necessary.
>
> thanks
> Holger
>