You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@karaf.apache.org by Philipp Hoenisch <ph...@gmail.com> on 2014/04/03 15:02:33 UTC
Native library and System.load
Hi everyone,
I'm a bit stuck getting a native library working in Karaf so I hope you guys
can help me:
Short Intro:
In my project I want to make use of lp_solve (a Mixed Integer Linear
Programming) for that I found some java libraries which make use of JNI to
access this library. This works just fine in a non-OSGi project, however, I
have some troubles to get it running within karaf.
The problem:
First the JavaLibrary tries to load the .so (or dll file for windows) via
System.loadLibrary("xxx), if that fails it extracts the .so (for or dll for
windows) from a jar file and loads it via System.load(xxx). The second
approach works also in karaf (if I use the absolut path), but if the bundle
gets reloaded (or uninstalled and newly installed) I get an error message
such as: liblpsolve55j_x64.so already loaded in another classloader.
Which means I can only load it once
Other solution?:
I found a blog-post ([1]) and some users in this forum explaining how to
make use of native-libraries in osgi, however, for me System.loadLibrary
never found that library, maybe you know what I am doing wrong?
<pom.xml>
...
<Bundle-NativeCode>
xxx-library-so-64.so;osname=linux;processor=x86-64
</Bundle-NativeCode>
...
</pom.xml>
any hints are appreciated and thank you in advance :)
best regards
Philipp
p.s. the bundle is self-contained, so I can upload it somewhere if wanted.
[1]http://holistictendencies.wordpress.com/2011/03/28/bundle-nativecode-using-platform-specific-dlls-from-osgi/
--
View this message in context: http://karaf.922171.n3.nabble.com/Native-library-and-System-load-tp4032562.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Native library and System.load
Posted by Achim Nierbeck <bc...@googlemail.com>.
yes, and maybe read this blog [1], especially the comments.
It drills down to the drawbacks of using Native-Code in OSGi.
Make sure you don't have to much dependencies :)
regards, Achim
[1] -
http://robertvarttinen.blogspot.de/2008/12/bundle-nativecode-in-osgi-manifest.html
2014-04-03 16:31 GMT+02:00 Jean-Baptiste Onofré <jb...@nanthrax.net>:
> I guess that Achim mentioned accessing directly to the bundle jar.
>
> Something like:
>
> static {
> try {
> System.loadLibrary("crypt"); // used for tests. This library in
> classpath only
> } catch (UnsatisfiedLinkError e) {
> try {
> NativeUtils.loadLibraryFromJar("/natives/crypt.dll"); //
> during runtime. .DLL within .JAR
> } catch (IOException e1) {
> throw new RuntimeException(e1);
>
> }
> }
> }
>
> On 04/03/2014 04:25 PM, Philipp Hoenisch wrote:
>
>> Can you explain that a bit more, i mean. how do I load a native library
>> via
>> the bundle loader?
>>
>>
>>
>>
>> --
>> View this message in context: http://karaf.922171.n3.nabble.
>> com/Native-library-and-System-load-tp4032562p4032568.html
>> Sent from the Karaf - User mailing list archive at Nabble.com.
>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
--
Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>
Software Architect / Project Manager / Scrum Master
Re: Native library and System.load
Posted by Achim Nierbeck <bc...@googlemail.com>.
Hi,
it's been a while I used this the last time.
Dll is just a synonym for any linked library :-)
Back to your questions, well as I pointed out, since libraries in general
can't be unloaded unless you destroy the classloader which is taken care by
the OSGi framework and only happens if there are no references to it
anymore.
That's why the library will be "renamed" inside your bundle. So to get a
hold of your library I can't remember right now, if
a) you need to load the library by the new name
b) still can do a loadLibrary as before.
That's the part you need to experiment.
But I'm certain the library will have some renaming to it, though I'm not
sure if there is some "magic symlink" to the library so a loadLibrary(name)
actually works ...
regards, Achim
2015-02-03 14:11 GMT+01:00 Philomatic <ma...@gottschaemmer.net>:
> Hi Achim,
>
> thanks for pointing out the issue with dynamic linked libraries, indeed I
> did not pay attention to the magic word "dynamic" in the blog post. My
> problem is currently that I'm not really into this topic, but have to deal
> with it from the Java side. Therefore, two questions:
>
> 1.
>
> We have only a single native library, so no reference problems between
> dynamic libs, like you've mentioned. In fact, we actually don't use .dll,
> but .so on linux arm, but that should be the same for loading in OSGi.
>
>
> > Therefore create a statically linked A.dll
>
> How can this be done? I thought a .dll is always dynamic linked?
>
> 2.
>
>
> > the name might need to be "Dynamic" as the container might rename the dll
> > because a dll can't be unloaded unless the corresponding class-loader is
> > destroyed. During this time the second native library will be renamed.
>
> Like said, there is no second native library which could be renamed.
> What do you mean by the "Dynamic" name?
>
> Best regards
> Patrick
>
>
>
>
>
> --
> View this message in context:
> http://karaf.922171.n3.nabble.com/Native-library-and-System-load-tp4032562p4038253.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
--
Apache Member
Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>
Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS>
Software Architect / Project Manager / Scrum Master
Re: Native library and System.load
Posted by Philomatic <ma...@gottschaemmer.net>.
Hi Achim,
thanks for pointing out the issue with dynamic linked libraries, indeed I
did not pay attention to the magic word "dynamic" in the blog post. My
problem is currently that I'm not really into this topic, but have to deal
with it from the Java side. Therefore, two questions:
1.
We have only a single native library, so no reference problems between
dynamic libs, like you've mentioned. In fact, we actually don't use .dll,
but .so on linux arm, but that should be the same for loading in OSGi.
> Therefore create a statically linked A.dll
How can this be done? I thought a .dll is always dynamic linked?
2.
> the name might need to be "Dynamic" as the container might rename the dll
> because a dll can't be unloaded unless the corresponding class-loader is
> destroyed. During this time the second native library will be renamed.
Like said, there is no second native library which could be renamed.
What do you mean by the "Dynamic" name?
Best regards
Patrick
--
View this message in context: http://karaf.922171.n3.nabble.com/Native-library-and-System-load-tp4032562p4038253.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Native library and System.load
Posted by Achim Nierbeck <bc...@googlemail.com>.
Hi,
the main issue is with dynamically loaded native libraries.
For example you have two dll's in your Bundle X, A.dll and B.dll where A
references B, this only works once.
With an update the dlls are supposed to be renamed to A2.dll and B2.dll
where the linkage of A to B doesn't work anymore.
Therefore create a statically linked A.dll
This doesn't matter for references to dlls outside the classloader scope,
dlls that reside in the system path.
For example if B.dll referencs X.dll which is in the system path it will
work.
To update bundles that contain native parts, make sure you have a static
linked librarie, this way it's updatable by osgi and just use
loadLibrary(name)
the name might need to be "Dynamic" as the container might rename the dll
because a dll can't be unloaded unless the corresponding class-loader is
destroyed. During this time the second native library will be renamed.
regards, Achim
2015-02-03 12:07 GMT+01:00 Philomatic <ma...@gottschaemmer.net>:
> Hi,
>
> (replying on this one, as it adresses exactly my problem.)
>
> We're also stuck with loading native libraries in an OSGi App in Karaf with
> Equinox as framework.
> Maybe I'm a little bit blindfolded, so far I've got the following:
>
> Our native lib is inside our bundle jar.
> We have declared it in our Manifest Header with:
>
> Bundle-NativeCode: lib/win_x86/RS485.dll;
> processor=x86;
> osname=win32,
> lib/linux_arm/libRS485.so;
> processor=arm;
> osname=linux,
> ...
>
> (We do not have this in our pom.xml, as our MANIFEST.MFs aren't
> auto-generated by maven. However, I'm wondering what the (side-)effects
> would be? )
>
> And in the Activator class:
>
> @Override
> public void start(BundleContext context) throws Exception {
>
> Activator.context = context;
>
> if (OSDetector.getOperatingSystemType().equals(OSType.Linux)) {
> System.loadLibrary("RS485");
> logger.info("RS485 native library loaded");
> RS485.setSimulation(false);
> } else {
> RS485.setSimulation(true);
> }
>
> (Are there any benefits by putting this into a static block instead of the
> Activator.start method?
> Plugin is a Singleton, ofc.)
>
> This works as a one-time solution:
> If I install and start the bundle in a clean Karaf installation, everything
> gets loaded and is working peachy.
> However, if I try to stop the bundle, uninstall it and install a new
> version, I'll get a Java LinkageError.
> Even after a Karaf restart, there's still the LinkageError and the bundle
> can't be loaded.
> (Note: If I update the bundle directly and restart karaf, this does not
> occur always! Will check further into this behavior.)
>
> So far, I understand this by the OSGi Spec and jbonofre's answer:
>
>
> jbonofre wrote
> > Which is normal, as the uninstall of a bundle doesn't unload the native
> > library.
> >
> > That's why, it would be better to put the dll/so in Karaf lib.
>
> However, I don't want to put the lib in karaf/lib folder, for two reasons:
> 1. I'd have to load it via System.loadlibrary(...) anyways, right? Which
> would lead me to the same behavior, as the lib does not get unloaded by a
> bundle stop?
>
> 2. I could not update the lib by updating our bundle. This is actually our
> goal, as the lib is a needed self development in C.
>
> What I don't understand is the "use the bundle loader to load native libs"
> thing, mentioning this piece of code by Achim and JB:
>
>
> > static {
> > try {
> > System.loadLibrary("crypt"); // used for tests. This library in
> > classpath only
> > } catch (UnsatisfiedLinkError e) {
> > try {
> > NativeUtils.loadLibraryFromJar("/natives/crypt.dll"); //
> > during runtime. .DLL within .JAR
> > } catch (IOException e1) {
> > throw new RuntimeException(e1);
> > }
> > }
> > }
>
> the try block works for us during runtime with limits, like mentioned
> above.
> But where does
>
> NativeUtils.loadLibraryFromJar("/natives/crypt.dll");
>
> come from? If you're referencing this
> <
> http://adamheinrich.com/blog/2012/how-to-load-native-jni-library-from-jar/
> >
> blogpost, I wonder:
>
> Why is this working? Where's the difference between loading with
> System.loadLibrary(...) and copying the file to a tempdir and loading via
> System.load(PathToTempFile)? Besides, Adan Heinrich states:
>
>
> > This approach is just fine if you have dynamic library outside the
> > application’s JAR archive, but when bundling dynamic library within the
> > JAR it is necessary to extract the library into filesystem before loading
> > it. And that’s exactly what my code does.
>
> However, loading directly from within the jar is at least possible in OSGi?
> No need for extracting to file system and wirking with absolute paths.
>
> tl;dr
> How can I (re-)load successfully our native lib during runtime with update
> possibility?
>
> Best regards
>
>
>
> --
> View this message in context:
> http://karaf.922171.n3.nabble.com/Native-library-and-System-load-tp4032562p4038250.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
--
Apache Member
Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/> Committer &
Project Lead
blog <http://notizblog.nierbeck.de/>
Co-Author of Apache Karaf Cookbook <http://bit.ly/1ps9rkS>
Software Architect / Project Manager / Scrum Master
Re: Native library and System.load
Posted by Philomatic <ma...@gottschaemmer.net>.
Hi,
(replying on this one, as it adresses exactly my problem.)
We're also stuck with loading native libraries in an OSGi App in Karaf with
Equinox as framework.
Maybe I'm a little bit blindfolded, so far I've got the following:
Our native lib is inside our bundle jar.
We have declared it in our Manifest Header with:
Bundle-NativeCode: lib/win_x86/RS485.dll;
processor=x86;
osname=win32,
lib/linux_arm/libRS485.so;
processor=arm;
osname=linux,
...
(We do not have this in our pom.xml, as our MANIFEST.MFs aren't
auto-generated by maven. However, I'm wondering what the (side-)effects
would be? )
And in the Activator class:
@Override
public void start(BundleContext context) throws Exception {
Activator.context = context;
if (OSDetector.getOperatingSystemType().equals(OSType.Linux)) {
System.loadLibrary("RS485");
logger.info("RS485 native library loaded");
RS485.setSimulation(false);
} else {
RS485.setSimulation(true);
}
(Are there any benefits by putting this into a static block instead of the
Activator.start method?
Plugin is a Singleton, ofc.)
This works as a one-time solution:
If I install and start the bundle in a clean Karaf installation, everything
gets loaded and is working peachy.
However, if I try to stop the bundle, uninstall it and install a new
version, I'll get a Java LinkageError.
Even after a Karaf restart, there's still the LinkageError and the bundle
can't be loaded.
(Note: If I update the bundle directly and restart karaf, this does not
occur always! Will check further into this behavior.)
So far, I understand this by the OSGi Spec and jbonofre's answer:
jbonofre wrote
> Which is normal, as the uninstall of a bundle doesn't unload the native
> library.
>
> That's why, it would be better to put the dll/so in Karaf lib.
However, I don't want to put the lib in karaf/lib folder, for two reasons:
1. I'd have to load it via System.loadlibrary(...) anyways, right? Which
would lead me to the same behavior, as the lib does not get unloaded by a
bundle stop?
2. I could not update the lib by updating our bundle. This is actually our
goal, as the lib is a needed self development in C.
What I don't understand is the "use the bundle loader to load native libs"
thing, mentioning this piece of code by Achim and JB:
> static {
> try {
> System.loadLibrary("crypt"); // used for tests. This library in
> classpath only
> } catch (UnsatisfiedLinkError e) {
> try {
> NativeUtils.loadLibraryFromJar("/natives/crypt.dll"); //
> during runtime. .DLL within .JAR
> } catch (IOException e1) {
> throw new RuntimeException(e1);
> }
> }
> }
the try block works for us during runtime with limits, like mentioned above.
But where does
NativeUtils.loadLibraryFromJar("/natives/crypt.dll");
come from? If you're referencing this
<http://adamheinrich.com/blog/2012/how-to-load-native-jni-library-from-jar/>
blogpost, I wonder:
Why is this working? Where's the difference between loading with
System.loadLibrary(...) and copying the file to a tempdir and loading via
System.load(PathToTempFile)? Besides, Adan Heinrich states:
> This approach is just fine if you have dynamic library outside the
> application’s JAR archive, but when bundling dynamic library within the
> JAR it is necessary to extract the library into filesystem before loading
> it. And that’s exactly what my code does.
However, loading directly from within the jar is at least possible in OSGi?
No need for extracting to file system and wirking with absolute paths.
tl;dr
How can I (re-)load successfully our native lib during runtime with update
possibility?
Best regards
--
View this message in context: http://karaf.922171.n3.nabble.com/Native-library-and-System-load-tp4032562p4038250.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Native library and System.load
Posted by Philipp Hoenisch <ph...@gmail.com>.
@JB,
thank you for clarification,
it works that way in combination with the stuff below
@Achim
I just came across that blog-post as well, but I couldn't get the
<Bundle-NativeCode>.... working,.... at first...
but now I found out that it was just a naming problem:
At the beginning my library's name was lpsolve55j_x64.so (for x64), but the
java library loader was not able to find it by calling
System.loadLibrary(xx), because this command expects that the library starts
with lib (not 100% sure :) ), then I renamed that file as it was and tried
<Bundle-NativeCode>
liblpsolve55j_x64.so;osname=linux;processor=x86-64
</Bundle-NativeCode>
tada, works like a charm :) even with uninstalling and reinstalling :)
thank you guys for your help
cheers
Philipp
--
View this message in context: http://karaf.922171.n3.nabble.com/Native-library-and-System-load-tp4032562p4032571.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Native library and System.load
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
I guess that Achim mentioned accessing directly to the bundle jar.
Something like:
static {
try {
System.loadLibrary("crypt"); // used for tests. This library in
classpath only
} catch (UnsatisfiedLinkError e) {
try {
NativeUtils.loadLibraryFromJar("/natives/crypt.dll"); //
during runtime. .DLL within .JAR
} catch (IOException e1) {
throw new RuntimeException(e1);
}
}
}
On 04/03/2014 04:25 PM, Philipp Hoenisch wrote:
> Can you explain that a bit more, i mean. how do I load a native library via
> the bundle loader?
>
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Native-library-and-System-load-tp4032562p4032568.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Native library and System.load
Posted by Philipp Hoenisch <ph...@gmail.com>.
Can you explain that a bit more, i mean. how do I load a native library via
the bundle loader?
--
View this message in context: http://karaf.922171.n3.nabble.com/Native-library-and-System-load-tp4032562p4032568.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Native library and System.load
Posted by Achim Nierbeck <bc...@googlemail.com>.
Which isn't the only solution. With native libs you should use the bundle
loafer to load native libs. With this it is possible to update also bundles
with native code.
Regards, Achim
sent from mobile device
Am 03.04.2014 15:38 schrieb "Jean-Baptiste Onofré" <jb...@nanthrax.net>:
> Which is normal, as the uninstall of a bundle doesn't unload the native
> library.
>
> That's why, it would be better to put the dll/so in Karaf lib.
>
> Regards
> JB
>
> On 04/03/2014 03:34 PM, Philipp Hoenisch wrote:
>
>> haha, that was to obvious right? :)
>>
>> well, at least System.loadLibrary(xxx) works now,
>> but still, only one time, i.e. if I uninstall and reinstall that bundle, I
>> receive again the:
>> java.lang.UnsatisfiedLinkError: ... xxxx.so already loaded in another
>> classloader Exception
>>
>>
>>
>>
>>
>>
>>
>> --
>> View this message in context: http://karaf.922171.n3.nabble.
>> com/Native-library-and-System-load-tp4032562p4032564.html
>> Sent from the Karaf - User mailing list archive at Nabble.com.
>>
>>
> --
> Jean-Baptiste Onofré
> jbonofre@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
Re: Native library and System.load
Posted by Philipp Hoenisch <ph...@gmail.com>.
ah, that explains the "reloading" problem.
But how do I load that library then? I mean, I copied it to the dll/so into
the Karaf lib folder, but I still have to call System.loadLibrary right?
--
View this message in context: http://karaf.922171.n3.nabble.com/Native-library-and-System-load-tp4032562p4032566.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Native library and System.load
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Which is normal, as the uninstall of a bundle doesn't unload the native
library.
That's why, it would be better to put the dll/so in Karaf lib.
Regards
JB
On 04/03/2014 03:34 PM, Philipp Hoenisch wrote:
> haha, that was to obvious right? :)
>
> well, at least System.loadLibrary(xxx) works now,
> but still, only one time, i.e. if I uninstall and reinstall that bundle, I
> receive again the:
> java.lang.UnsatisfiedLinkError: ... xxxx.so already loaded in another
> classloader Exception
>
>
>
>
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Native-library-and-System-load-tp4032562p4032564.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com
Re: Native library and System.load
Posted by Philipp Hoenisch <ph...@gmail.com>.
haha, that was to obvious right? :)
well, at least System.loadLibrary(xxx) works now,
but still, only one time, i.e. if I uninstall and reinstall that bundle, I
receive again the:
java.lang.UnsatisfiedLinkError: ... xxxx.so already loaded in another
classloader Exception
--
View this message in context: http://karaf.922171.n3.nabble.com/Native-library-and-System-load-tp4032562p4032564.html
Sent from the Karaf - User mailing list archive at Nabble.com.
Re: Native library and System.load
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Phillip,
did you try to just put the dll/so in lib folder of Karaf ?
Regards
JB
On 04/03/2014 03:02 PM, Philipp Hoenisch wrote:
> Hi everyone,
>
> I'm a bit stuck getting a native library working in Karaf so I hope you guys
> can help me:
>
> Short Intro:
> In my project I want to make use of lp_solve (a Mixed Integer Linear
> Programming) for that I found some java libraries which make use of JNI to
> access this library. This works just fine in a non-OSGi project, however, I
> have some troubles to get it running within karaf.
>
> The problem:
> First the JavaLibrary tries to load the .so (or dll file for windows) via
> System.loadLibrary("xxx), if that fails it extracts the .so (for or dll for
> windows) from a jar file and loads it via System.load(xxx). The second
> approach works also in karaf (if I use the absolut path), but if the bundle
> gets reloaded (or uninstalled and newly installed) I get an error message
> such as: liblpsolve55j_x64.so already loaded in another classloader.
> Which means I can only load it once
>
> Other solution?:
> I found a blog-post ([1]) and some users in this forum explaining how to
> make use of native-libraries in osgi, however, for me System.loadLibrary
> never found that library, maybe you know what I am doing wrong?
>
> <pom.xml>
> ...
> <Bundle-NativeCode>
> xxx-library-so-64.so;osname=linux;processor=x86-64
> </Bundle-NativeCode>
> ...
> </pom.xml>
>
> any hints are appreciated and thank you in advance :)
>
> best regards
> Philipp
>
> p.s. the bundle is self-contained, so I can upload it somewhere if wanted.
>
> [1]http://holistictendencies.wordpress.com/2011/03/28/bundle-nativecode-using-platform-specific-dlls-from-osgi/
>
>
>
> --
> View this message in context: http://karaf.922171.n3.nabble.com/Native-library-and-System-load-tp4032562.html
> Sent from the Karaf - User mailing list archive at Nabble.com.
>
--
Jean-Baptiste Onofré
jbonofre@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com