You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by David Daeschler <da...@gmail.com> on 2014/05/27 05:33:50 UTC
Strange trouble with JNA on CentOS6
Good evening list,
I'm having a rather confusing issue with 4 new nodes that I put up. JNA
doesn't seem to want to work in production even though it works on my test
VM that should have pretty much the same setup.
I have created a series of setup scripts to bring up new nodes. I am using
a tested and tried configuration that has been running very well in
production for a while and in all cases the following holds true:
OS: CentOS6 with latest updates 2.6.32-431.17.1.el6.x86_64
Java: Java(TM) SE Runtime Environment (build 1.6.0_30-b12)
Cassandra: 1.0.12
JNA: Tried both jna-3.5.1.jar, and jna-4.1.0.jar
In both my old production servers and my test setup (which I can recreate
over and over and get the same results), startup yields:
"CLibrary.java (line 109) JNA mlockall successful"
However, for some reason on the NEW production boxes, I get:
"Unable to link C library. Native methods will be disabled"
The JNA libs are exact copies and part of our setup scripts.
There are only 2 major differences I can think of. The first is the
hardware. All are 64 bit platforms, but the newer servers has 48 GB of ram
and newer processors than the older boxes and my test vm. I don't think
that would have anything to do with it, but it's worth mentioning. Both
production sets are Xeons though.
The second difference is that the new image would've been set up by
Rackspace while my test uses an older image and brings it up to date with
yum. However, as part of setup I apply yum update to all of them, so they
should be running at least the same versions of whatever they have in
common.
Another interesting note is that while looking for "jna" on the disk, I
also see what looks like an ELF binary in /tmp named
/tmp/jna-1073564104/jna4691935553862497129.tmp.
Thank you ahead of time for any help you may be able to offer.
--
David Daeschler
Re: Strange trouble with JNA on CentOS6
Posted by da...@gmail.com.
Great! Thanks for the feedback gents.
Have a nice week.
From: Nate McCall
Sent: Tuesday, May 27, 2014 12:39 PM
To: Cassandra Users
I missed that - thanks Holger!
On Tue, May 27, 2014 at 10:14 AM, Holger Hoffstätte <ho...@googlemail.com> wrote:
On Tue, 27 May 2014 09:26:41 -0500, Nate McCall wrote:
>> In the new setup /tmp was mounted as noexec. So it looks like JNA was
>> putting the native library here and then it was unable to execute it.
>>
>>
> If you wanted to keep /tmp noexec (not a bad idea depending on
> environment) you could also append the following to cassandra-env.sh:
>
> JVM_OPTS="$JVM_OPTS -Djava.io.tmpdir=/some/other/dir"
Recent versions of JNA have a dedicated property "jna.tmpdir" for this,
so that files that are supposed to go into /tmp (i.e. regular temp files)
are unaffected. It has precedence over java.io.tmpdir.
-h
--
-----------------
Nate McCall
Austin, TX
@zznate
Co-Founder & Sr. Technical Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com
Re: Strange trouble with JNA on CentOS6
Posted by Nate McCall <na...@thelastpickle.com>.
I missed that - thanks Holger!
On Tue, May 27, 2014 at 10:14 AM, Holger Hoffstätte <
holger.hoffstaette@googlemail.com> wrote:
> On Tue, 27 May 2014 09:26:41 -0500, Nate McCall wrote:
>
> >> In the new setup /tmp was mounted as noexec. So it looks like JNA was
> >> putting the native library here and then it was unable to execute it.
> >>
> >>
> > If you wanted to keep /tmp noexec (not a bad idea depending on
> > environment) you could also append the following to cassandra-env.sh:
> >
> > JVM_OPTS="$JVM_OPTS -Djava.io.tmpdir=/some/other/dir"
>
> Recent versions of JNA have a dedicated property "jna.tmpdir" for this,
> so that files that are supposed to go into /tmp (i.e. regular temp files)
> are unaffected. It has precedence over java.io.tmpdir.
>
> -h
>
>
--
-----------------
Nate McCall
Austin, TX
@zznate
Co-Founder & Sr. Technical Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com
Re: Strange trouble with JNA on CentOS6
Posted by Holger Hoffstätte <ho...@googlemail.com>.
On Tue, 27 May 2014 09:26:41 -0500, Nate McCall wrote:
>> In the new setup /tmp was mounted as noexec. So it looks like JNA was
>> putting the native library here and then it was unable to execute it.
>>
>>
> If you wanted to keep /tmp noexec (not a bad idea depending on
> environment) you could also append the following to cassandra-env.sh:
>
> JVM_OPTS="$JVM_OPTS -Djava.io.tmpdir=/some/other/dir"
Recent versions of JNA have a dedicated property "jna.tmpdir" for this,
so that files that are supposed to go into /tmp (i.e. regular temp files)
are unaffected. It has precedence over java.io.tmpdir.
-h
Re: Strange trouble with JNA on CentOS6
Posted by Nate McCall <na...@thelastpickle.com>.
>
>
> In the new setup /tmp was mounted as noexec. So it looks like JNA was
> putting the native library here and then it was unable to execute it.
>
If you wanted to keep /tmp noexec (not a bad idea depending on environment)
you could also append the following to cassandra-env.sh:
JVM_OPTS="$JVM_OPTS -Djava.io.tmpdir=/some/other/dir"
--
-----------------
Nate McCall
Austin, TX
@zznate
Co-Founder & Sr. Technical Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com
Re: Strange trouble with JNA on CentOS6
Posted by David Daeschler <da...@gmail.com>.
Figured this one out..
In the new setup /tmp was mounted as noexec. So it looks like JNA was
putting the native library here and then it was unable to execute it.
I hope this can help someone else.
On Mon, May 26, 2014 at 11:33 PM, David Daeschler <david.daeschler@gmail.com
> wrote:
> Good evening list,
>
> I'm having a rather confusing issue with 4 new nodes that I put up. JNA
> doesn't seem to want to work in production even though it works on my test
> VM that should have pretty much the same setup.
>
> I have created a series of setup scripts to bring up new nodes. I am using
> a tested and tried configuration that has been running very well in
> production for a while and in all cases the following holds true:
>
> OS: CentOS6 with latest updates 2.6.32-431.17.1.el6.x86_64
> Java: Java(TM) SE Runtime Environment (build 1.6.0_30-b12)
> Cassandra: 1.0.12
> JNA: Tried both jna-3.5.1.jar, and jna-4.1.0.jar
>
> In both my old production servers and my test setup (which I can recreate
> over and over and get the same results), startup yields:
>
> "CLibrary.java (line 109) JNA mlockall successful"
>
> However, for some reason on the NEW production boxes, I get:
>
> "Unable to link C library. Native methods will be disabled"
>
>
> The JNA libs are exact copies and part of our setup scripts.
>
> There are only 2 major differences I can think of. The first is the
> hardware. All are 64 bit platforms, but the newer servers has 48 GB of ram
> and newer processors than the older boxes and my test vm. I don't think
> that would have anything to do with it, but it's worth mentioning. Both
> production sets are Xeons though.
>
> The second difference is that the new image would've been set up by
> Rackspace while my test uses an older image and brings it up to date with
> yum. However, as part of setup I apply yum update to all of them, so they
> should be running at least the same versions of whatever they have in
> common.
>
> Another interesting note is that while looking for "jna" on the disk, I
> also see what looks like an ELF binary in /tmp named
> /tmp/jna-1073564104/jna4691935553862497129.tmp.
>
>
> Thank you ahead of time for any help you may be able to offer.
>
> --
> David Daeschler
>
--
David Daeschler