You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Hiram Chirino (JIRA)" <ji...@apache.org> on 2010/12/27 22:24:48 UTC
[jira] Created: (CASSANDRA-1908) Implement the CLibrary using JNI
module to avoid the LGPL dependency on JNA
Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA
---------------------------------------------------------------------------
Key: CASSANDRA-1908
URL: https://issues.apache.org/jira/browse/CASSANDRA-1908
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Hiram Chirino
Fix For: 0.7.1
Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1908) Implement the CLibrary using JNI
module to avoid the LGPL dependency on JNA
Posted by "Hiram Chirino (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hiram Chirino updated CASSANDRA-1908:
-------------------------------------
Attachment: cassandra-jni.zip
Attaching a port of the latest CLibrary. Build with maven.
It code generates the JNI library implementation. See readme.md for more details.
> Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-1908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1908
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Hiram Chirino
> Fix For: 0.7.1
>
> Attachments: cassandra-jni.zip
>
>
> Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Resolved] (CASSANDRA-1908) Implement the CLibrary using JNI
module to avoid the LGPL dependency on JNA
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-1908.
---------------------------------------
Resolution: Not A Problem
As above, we're pretty happy with how JNA is working for us.
> Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-1908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1908
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Hiram Chirino
> Attachments: cassandra-jni.zip
>
>
> Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-1908) Implement the CLibrary using JNI
module to avoid the LGPL dependency on JNA
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975351#action_12975351 ]
Jonathan Ellis commented on CASSANDRA-1908:
-------------------------------------------
Thanks, Hiram!
Personally, I prefer the inconvenience of an LGPL library (which is largely mitigated by deb/rpm packaging) to the inconvenience of maintaining a maven + autoconf build for a native extension, but I'm open to exploring the options.
> Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-1908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1908
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Hiram Chirino
> Fix For: 0.7.1
>
> Attachments: cassandra-jni.zip
>
>
> Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-1908) Implement the
CLibrary using JNI module to avoid the LGPL dependency on JNA
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975512#action_12975512 ]
Eric Evans edited comment on CASSANDRA-1908 at 12/28/10 10:21 AM:
------------------------------------------------------------------
It might be trivial to implement a JNI library, but I don't think it's going to be trivial to build architecture dependent code at release time, field issues, debug, etc, for N platforms.
I'm also open to alternatives, but I'd need to be convinced that the cure wasn't worse than the disease.
was (Author: urandom):
It might be trivial to implement a JNI library, but I don't think it's going to be trivial to build architecture dependent code at release time, field issues, debug, etc, for N platforms.
I'm also open to alternatives, but I'd need to be convinced that the cure wasn't worth the disease.
> Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-1908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1908
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Hiram Chirino
> Fix For: 0.7.1
>
> Attachments: cassandra-jni.zip
>
>
> Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1908) Implement the CLibrary using JNI
module to avoid the LGPL dependency on JNA
Posted by "Peter Schuller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975641#action_12975641 ]
Peter Schuller commented on CASSANDRA-1908:
-------------------------------------------
I was thinking along those lines (separate project) before when there was some JNA vs. JNI debate w.r.t. direct I/O. A separate project could that provides some fairly simple and specific things that tend to be useful in a pragmatic way, without trying to be overly formal or a complete posix wrapper. It could be useful for others except Cassandra, and it would remove any build hassle trade-offs in the JNA vs. JNI decision from Cassandra.
Also, for future work that might imply very frequent mincore() calls, the calling overhead, if the numbers claimed are correct, could possibly be significant for Cassandra (although I'm pretty paranoid about the cost of mincore() to begin with, which might dwarf the JNI vs. JNA issue).
> Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-1908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1908
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Hiram Chirino
> Fix For: 0.7.1
>
> Attachments: cassandra-jni.zip
>
>
> Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1908) Implement the CLibrary using JNI
module to avoid the LGPL dependency on JNA
Posted by "Stephen Connolly (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983182#action_12983182 ]
Stephen Connolly commented on CASSANDRA-1908:
---------------------------------------------
You could always host the JNI project on http://apache-extras.org/ and as long as it uses ASLv2 and if it publishes to Maven Central it should be possible to pull the build artifacts down... that would decouple the build and release of the JNI code from the build and release of apache cassandra
> Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-1908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1908
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Hiram Chirino
> Fix For: 0.7.1
>
> Attachments: cassandra-jni.zip
>
>
> Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1908) Implement the CLibrary using JNI
module to avoid the LGPL dependency on JNA
Posted by "Hiram Chirino (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975603#action_12975603 ]
Hiram Chirino commented on CASSANDRA-1908:
------------------------------------------
The JNA solution has all the same architecture dependent code for N platform problems, what's worse is that it's using a FFI and your hoping that everything maps out right.
Additionally, because of the FFI, the JNI interface is going to be orders of magnitude more efficent. See: https://jna.dev.java.net/#performance
But yeah, if you want to include a few pre-built JNI libraries, for example linux 32/64 and OS X, it's release process gets more complicated as you now have to build the jni modules on a couple of platforms. To keep that effort to minimum I'd recommend you guys independently release the jni module and then just include it as part of the cassandra distribution. Once it stabilizes, you probably won't be releasing it very often.
> Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-1908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1908
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Hiram Chirino
> Fix For: 0.7.1
>
> Attachments: cassandra-jni.zip
>
>
> Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1908) Implement the CLibrary using JNI
module to avoid the LGPL dependency on JNA
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975512#action_12975512 ]
Eric Evans commented on CASSANDRA-1908:
---------------------------------------
It might be trivial to implement a JNI library, but I don't think it's going to be trivial to build architecture dependent code at release time, field issues, debug, etc, for N platforms.
I'm also open to alternatives, but I'd need to be convinced that the cure wasn't worth the disease.
> Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-1908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1908
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Hiram Chirino
> Fix For: 0.7.1
>
> Attachments: cassandra-jni.zip
>
>
> Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1908) Implement the CLibrary using JNI
module to avoid the LGPL dependency on JNA
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975802#action_12975802 ]
Jonathan Ellis commented on CASSANDRA-1908:
-------------------------------------------
The page Hiram linked says "JNA supports a direct mapping method which can improve performance substantially, approaching that of custom JNI." This is the approach we are using (Native.register, instead of the slower Native.loadLibrary).
> Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-1908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1908
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Hiram Chirino
> Fix For: 0.7.1
>
> Attachments: cassandra-jni.zip
>
>
> Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1908) Implement the CLibrary using JNI
module to avoid the LGPL dependency on JNA
Posted by "Hiram Chirino (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975814#action_12975814 ]
Hiram Chirino commented on CASSANDRA-1908:
------------------------------------------
JNA does not really do the work of validating that the native method are properly mapped. Only testing on the actual platforms can do that. JNA just helps avoid pain of doing muti-platform builds (which is nice), but if you want to test on the actual platforms then you might as well do a platform specific build too.
The performance of the interface is only going to be critical if it's in a performance hot spot which right now does not seem to be the case.
> Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-1908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1908
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Hiram Chirino
> Fix For: 0.7.1
>
> Attachments: cassandra-jni.zip
>
>
> Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1908) Implement the CLibrary using JNI
module to avoid the LGPL dependency on JNA
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975799#action_12975799 ]
Eric Evans commented on CASSANDRA-1908:
---------------------------------------
bq. The JNA solution has all the same architecture dependent code for N platform problems, what's worse is that it's using a FFI and your hoping that everything maps out right.
Sure, but someone else is doing that work for us. And as Jonathan mentioned, JNA is package for all architectures on the most common platforms.
bq. Additionally, because of the FFI, the JNI interface is going to be orders of magnitude more efficent. See: https://jna.dev.java.net/#performance
Performance is always a good argument to make. Do you have before/after numbers to show that it will make a substantial difference for us?
> Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-1908
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1908
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Hiram Chirino
> Fix For: 0.7.1
>
> Attachments: cassandra-jni.zip
>
>
> Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.