You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Andy Tolbert (JIRA)" <ji...@apache.org> on 2016/03/12 22:37:03 UTC
[jira] [Comment Edited] (CASSANDRA-10735) Support netty openssl
(netty-tcnative) for client encryption
[ https://issues.apache.org/jira/browse/CASSANDRA-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15191824#comment-15191824 ]
Andy Tolbert edited comment on CASSANDRA-10735 at 3/12/16 9:36 PM:
-------------------------------------------------------------------
I went ahead and updated my [branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-10735] today to rebase it off of trunk and add a commit that enables using netty openssl in cassandra stress, example usage:
{code}
tools/bin/cassandra-stress write n=10000000 -rate threads=250 -transport ssl-openssl ssl-private-key=client.pem ssl-cert=client.crt truststore=client.truststore truststore-password=cassandra
{code}
The performance characteristics were pretty much exactly the same as I had observed previously, with exception to the no ssl run which had a few pauses (didn't read too much into it as I was focusing on the SSL configs).
There are a few caveats / things I wanted to mention:
# The cipher suite format for openssl is different than the format jdk ssl expects. Netty appropriately does translation when you provide the ciphers to convert the jdk-style cipher names into what openssl expects, but there is no way to convert it going back. {{SSLServerSocket.getSupportedCipherSuites()}} appears to return the open ssl style. This makes things tricky as {{SSLFactory.filterCipherSuites}} compares cipher's on the socket to those in our configuration, which currently use the JDK format. I think this becomes a non-issue if [~spodxx@gmail.com]'s [CASSANDRA-10508] is implemented. You can currently work around it by providing openssl cipher suites in your cassandra.yaml under client_encryption_options (see [this|https://gist.github.com/tolbertam/87267fa5708d998b57ee] for a list).
# One thing that is interesting to me is that netty's openssl builder will allow you to use a truststore for establishing trust, but not a keystore for your private keys. It looks like it does some work extracting items from the truststore into X509 certs. Because of this we need separate options for private key & cert (for openssl) and keystore (for jdk). I wonder if something similar can be done for keys in netty, will pursue this.
# The transport encryption options do not let you override the keystore and keystore password used for jdk ssl, it just uses defaults (conf/.keystore and cassandra for password). Would be good to add those as extra options, will add commit for that.
# With the version of netty we are using here (4.0.33) it seems the most recent version of netty-tcnative that works is [1.1.33.Fork11|http://central.maven.org/maven2/io/netty/netty-tcnative/1.1.33.Fork11/]. I ran into problems using Fork14 on the client side ([issue|https://github.com/netty/netty-tcnative/issues/124]), but it seems to work well on the server side. For openssl to work, download the appropriate platform jar and dump it in cassandra's lib directory.
# We could consider including the netty-tcnative jars in the lib directory as part of the distribution. The only problem would be where/if we have multiple jars that could match the same platform. For example, in Fork11 there are separate jars for fedora x64 and linux x64 and the presence of both in the classpath can be a problem.
Attached [^netty-ssl-trunk.tgz] which includes stress logs, example keystores/certs/pems/etc., and the html benchmark report (doesn't work in chrome for some reason).
!sslbench12-03.png!
(Red: no encryption, Orange: openssl on both ends, Purple: openssl client, jdk server, Green: jdk client jdk server, Blue: jdk client, openssl server
was (Author: andrew.tolbert):
I went ahead and updated my [branch|https://github.com/tolbertam/cassandra/tree/CASSANDRA-10735] today to rebase it off of trunk and add a commit that enables using netty openssl in cassandra stress, example usage:
{code}
tools/bin/cassandra-stress write n=10000000 -rate threads=250 -transport ssl-openssl ssl-private-key=client.pem ssl-cert=client.crt truststore=client.truststore truststore-password=cassandra
{code}
The performance characteristics were pretty much exactly the same as I had observed previously, with exception to the no ssl run which had a few pauses (didn't read too much into it as I was focusing on the SSL configs).
There are a few caveats / things I wanted to mention:
# The cipher suite format for openssl is different than the format jdk ssl expects. Netty appropriately does translation when you provide the ciphers to convert the jdk-style cipher names into what openssl expects, but there is no way to convert it going back. {{SSLServerSocket.getSupportedCipherSuites()}} appears to return the open ssl style. This makes things tricky as {{SSLFactory.filterCipherSuites}} compares cipher's on the socket to those in our configuration, which currently use the JDK format. I think this becomes a non-issue if [~spodxx@gmail.com]'s [CASSANDRA-10508] is implemented. You can currently work around it by providing openssl cipher suites in your cassandra.yaml under client_encryption_options (see [this|https://gist.github.com/tolbertam/87267fa5708d998b57ee] for a list).
# One thing that is interesting to me is that netty's openssl builder will allow you to use a truststore for establishing trust, but not a keystore for your private keys. It looks like it does some work extracting items from the truststore into X509 certs. Because of this we need separate options for private key & cert (for openssl) and keystore (for jdk). I wonder if something similar can be done for keys in netty, will pursue this.
# The transport encryption options do not let you override the keystore and keystore password used for jdk ssl, it just uses defaults (conf/.keystore and cassandra for password). Would be good to add those as extra options, will add commit for that.
# With the version of netty we are using here (4.0.33) it seems the most recent version of netty-tcnative that works is [1.1.33.Fork11|http://central.maven.org/maven2/io/netty/netty-tcnative/1.1.33.Fork11/]. I ran into problems using Fork14 on the client side ([issue|https://github.com/netty/netty-tcnative/issues/124]), but it seems to work well on the server side. For openssl to work, download the appropriate platform jar and dump it in cassandra's lib directory.
# We could consider including the netty-tcnative jars in the lib directory as part of the distribution. The only problem would be where/if we have multiple jars that could match the same platform. For example, in Fork11 there are separate jars for fedora x64 and linux x64 and the presence of both in the classpath can be a problem.
Attached [^netty-ssl-trunk.tgz] which includes stress logs, example keystores/certs/pems/etc., and the html benchmark report (doesn't work in chrome for some reason).
!sslbench12-03.png|thumbnail!
(Red: no encryption, Orange: openssl on both ends, Purple: openssl client, jdk server, Green: jdk client jdk server, Blue: jdk client, openssl server
> Support netty openssl (netty-tcnative) for client encryption
> ------------------------------------------------------------
>
> Key: CASSANDRA-10735
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10735
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Andy Tolbert
> Assignee: Aleksey Yeschenko
> Fix For: 3.x
>
> Attachments: netty-ssl-trunk.tgz, nettyssl-bench.tgz, nettysslbench.png, nettysslbench_small.png, sslbench12-03.png
>
>
> The java-driver recently added support for using netty openssl via [netty-tcnative|http://netty.io/wiki/forked-tomcat-native.html] in [JAVA-841|https://datastax-oss.atlassian.net/browse/JAVA-841], this shows a very measured improvement (numbers incoming on that ticket). It seems likely that this can offer improvement if implemented C* side as well.
> Since netty-tcnative has platform specific requirements, this should not be made the default, but rather be an option that one can use.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)