unable to find my native libraries on task node via JNA

[Julian Bui <ju...@gmail.com>]

Hi all,

I am hoping for someone with JNI or JNA experience on hadoop to help me.
 JNA execution on the tasks nodes appears to fail because it cannot find
the native libraries.  I can use the API to confirm that the native
libraries are in fact there.

I have set my jna.library.path to /user/bui/lib/ and I have set the
mapred.map.child.env to specify a jna.library.path in the children, and I
verified that my native libraries are in that directory and I have even put
my native library into the DistributedCache.  However, when JNA tries to
find it, then it just fails with the runtime error "Error: Unable to load
library 'memfxns': libmemfxns.so: cannot open shared object file: No such
file or directory"

Some code is attached.

I am wondering if there is a permissions issue or something related to .so
files needing to be set w/ execute flag (I know this cannot be done on
hdfs).

Any help would be greatly appreciated.

-Julian

In my hadoop application main method:

job.getConfiguration().set("jna.library.path", "/user/bui/lib/");
job.getConfiguration().set("mapred.map.child.env",
"jna.library.path=/user/bui/lib");

DistributedCache.addCacheFile(new
Path("/user/bui/lib/libmemfxns.so").toUri(), job.getConfiguration());
if(fs.exists(new Path("/user/bui/lib/libmemfxns.so")))
{
    System.out.println("/user/bui/lib/libmemfxns.so found in main"); // <--
this gets printed
}
========================================

In my hadoop application's reducer:

if(fs.exists(new Path("/user/bui/lib/libmemfxns.so")))
{
    System.out.println("/user/bui/lib/libmemfxns.so found in reducer"); //
<-- this also gets printed
}
JnaTest.getBuf(); // <-- crashes here, when it tries to instantiate the
native library

========================================

Inside JnaTest.getBuf():
MemFxnsLibrary blah = MemFxnsLibrary.INSTANCE;

========================================

Inside MemFxnsLibrary:
public interface MemFxnsLibrary extends Library
{
MemFxnsLibrary INSTANCE = (MemFxnsLibrary)Native.loadLibrary("memfxns",
MemFxnsLibrary.class);
 void getBuf(Pointer buf, Pointer length);
}


runtime error:
Error: Unable to load library 'memfxns': libmemfxns.so: cannot open shared
object file: No such file or directory