You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@kudu.apache.org by "Alexey Serbin (Code Review)" <ge...@cloudera.org> on 2021/03/16 02:54:42 UTC
[kudu-CR] WIP KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Alexey Serbin has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17189
Change subject: WIP KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
WIP KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
With this patch Kudu servers and Kudu C++ client can use TLSv1.3
for securing Kudu RPC communication. All tests run between C++
components now use TLSv1.3 when compiled with OpenSSL 1.1.1 and newer.
This patch also contains a new test to exercise TLSv1.3 handshake in
the context of establishing a connection for Kudu RPC.
WIP:
* add informative description for the changelist
* add run-time flag to control the list of TLSv1.3 ciphers
* pass the list of TLSv1.3 ciphers into TlsContext constructor
* add validator for newly added --rpc_tls_excluded_protocols flag
Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
---
M src/kudu/rpc/client_negotiation.cc
M src/kudu/rpc/messenger.cc
M src/kudu/rpc/messenger.h
M src/kudu/rpc/rpc-test.cc
M src/kudu/rpc/server_negotiation.cc
M src/kudu/security/tls_context.cc
M src/kudu/security/tls_context.h
M src/kudu/security/tls_handshake-test.cc
M src/kudu/security/tls_handshake.cc
M src/kudu/security/tls_handshake.h
M src/kudu/security/tls_socket.cc
M src/kudu/server/server_base.cc
12 files changed, 534 insertions(+), 106 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/89/17189/1
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 1
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
With this patch Kudu servers and Kudu C++ client can use TLSv1.3
for securing Kudu RPC communication. All tests run between C++
components now use TLSv1.3 when compiled with OpenSSL 1.1.1 or newer.
This patch introduces two new server-side flags:
--rpc_tls_ciphersuites
--rpc_tls_excluded_protocols
The former is to customize the list of preferred ciphers for TLSv1.3.
The latter is to control the set of TLS protocols used by Kudu servers
when securing connections for RPC communication. It can be used along
with the --rpc_tls_min_protocol flag to define the set of TLS protocols
available to Kudu servers when running a TLS handshake. For example,
set --rpc_tls_excluded_protocols=TLSv1.3 to exclude TLSv1.3 from
available options for securing Kudu RPC.
In addition, I added a validator for the --rpc_tls_min_protocol flag,
similar to the validator for --rpc_tls_excluded_protocols.
This patch also adds a new test to exercise TLSv1.3 handshake in
the context of establishing a connection for Kudu RPC.
The embedded webserver isn't updated yet with TLSv1.3 cipher controls.
That will be done in a separate patch because it requires pushing an
update to the squeasel webserver to introduce a new TLSv1.3-specific
flag and make a call to SSL_CTX_set_ciphersuites() correspondingly.
Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Reviewed-on: http://gerrit.cloudera.org:8080/17189
Tested-by: Alexey Serbin <as...@cloudera.com>
Reviewed-by: Grant Henke <gr...@apache.org>
---
M src/kudu/rpc/client_negotiation.cc
M src/kudu/rpc/messenger.cc
M src/kudu/rpc/messenger.h
M src/kudu/rpc/server_negotiation.cc
M src/kudu/security/security-test-util.cc
M src/kudu/security/security_flags.cc
M src/kudu/security/security_flags.h
M src/kudu/security/tls_context.cc
M src/kudu/security/tls_context.h
M src/kudu/security/tls_handshake-test.cc
M src/kudu/security/tls_handshake.cc
M src/kudu/security/tls_handshake.h
M src/kudu/server/server_base.cc
13 files changed, 529 insertions(+), 99 deletions(-)
Approvals:
Alexey Serbin: Verified
Grant Henke: Looks good to me, approved
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 11
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17189
to look at the new patch set (#2).
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
With this patch Kudu servers and Kudu C++ client can use TLSv1.3
for securing Kudu RPC communication. All tests run between C++
components now use TLSv1.3 when compiled with OpenSSL 1.1.1 or newer
versions.
This patch introduces two new server-side flags:
--rpc_tls_1_3_ciphers
--rpc_tls_excluded_protocols
The former is to customize the list of preferred ciphers for TLSv1.3.
The latter is to control the set of TLS protocols used by Kudu servers
when securing connections for RPC communication. It can be used along
with the --rpc_tls_min_protocol flag to define the set of TLS protocols
available to Kudu servers when running a TLS handshake. For example,
set --rpc_tls_excluded_protocols=TLSv1.3 to exclude TLSv1.3 from
available options for securing Kudu RPC.
In addition, I added a validator for the --rpc_tls_min_protocol flag,
similar to the validator for --rpc_tls_excluded_protocols.
This patch also adds a new test to exercise TLSv1.3 handshake in
the context of establishing a connection for Kudu RPC.
The embedded webserver isn't updated yet with TLSv1.3 cipher controls:
that will be done in a separate patch because it requires first
to squeasel to introduce a new TLSv1.3-specific flag and call
SSL_CTX_set_ciphersuites() correspondingly.
Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
---
M src/kudu/rpc/client_negotiation.cc
M src/kudu/rpc/messenger.cc
M src/kudu/rpc/messenger.h
M src/kudu/rpc/server_negotiation.cc
M src/kudu/security/security-test-util.cc
M src/kudu/security/security_flags.cc
M src/kudu/security/security_flags.h
M src/kudu/security/tls_context.cc
M src/kudu/security/tls_context.h
M src/kudu/security/tls_handshake-test.cc
M src/kudu/security/tls_handshake.cc
M src/kudu/security/tls_handshake.h
M src/kudu/server/server_base.cc
13 files changed, 463 insertions(+), 86 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/89/17189/2
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Attila Bukor, Kudu Jenkins, Andrew Wong, Grant Henke, Joe McDonnell,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17189
to look at the new patch set (#8).
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
With this patch Kudu servers and Kudu C++ client can use TLSv1.3
for securing Kudu RPC communication. All tests run between C++
components now use TLSv1.3 when compiled with OpenSSL 1.1.1 or newer
versions.
This patch introduces two new server-side flags:
--rpc_tls_1_3_ciphers
--rpc_tls_excluded_protocols
The former is to customize the list of preferred ciphers for TLSv1.3.
The latter is to control the set of TLS protocols used by Kudu servers
when securing connections for RPC communication. It can be used along
with the --rpc_tls_min_protocol flag to define the set of TLS protocols
available to Kudu servers when running a TLS handshake. For example,
set --rpc_tls_excluded_protocols=TLSv1.3 to exclude TLSv1.3 from
available options for securing Kudu RPC.
In addition, I added a validator for the --rpc_tls_min_protocol flag,
similar to the validator for --rpc_tls_excluded_protocols.
This patch also adds a new test to exercise TLSv1.3 handshake in
the context of establishing a connection for Kudu RPC.
The embedded webserver isn't updated yet with TLSv1.3 cipher controls.
That will be done in a separate patch because it requires pushing an
update to the squeasel webserver to introduce a new TLSv1.3-specific
flag and make a call to SSL_CTX_set_ciphersuites() correspondingly.
Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
---
M src/kudu/rpc/client_negotiation.cc
M src/kudu/rpc/messenger.cc
M src/kudu/rpc/messenger.h
M src/kudu/rpc/server_negotiation.cc
M src/kudu/security/security-test-util.cc
M src/kudu/security/security_flags.cc
M src/kudu/security/security_flags.h
M src/kudu/security/tls_context.cc
M src/kudu/security/tls_context.h
M src/kudu/security/tls_handshake-test.cc
M src/kudu/security/tls_handshake.cc
M src/kudu/security/tls_handshake.h
M src/kudu/server/server_base.cc
13 files changed, 493 insertions(+), 92 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/89/17189/8
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 8
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 5: Verified+1
unrelated test failure in TxnOpDispatcherITest.TxnWriteWhileReplicaDeleted
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 23 Mar 2021 21:28:46 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 10: Verified+1
unrelated test failure in DifferentFaultPoints/Kudu969Test.Test/1
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 10
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Wed, 07 Apr 2021 00:43:39 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 5:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17189/5/src/kudu/security/tls_context.cc
File src/kudu/security/tls_context.cc:
http://gerrit.cloudera.org:8080/#/c/17189/5/src/kudu/security/tls_context.cc@259
PS5, Line 259: #if OPENSSL_VERSION_NUMBER >= 0x10101000L
: // Set TLSv1.3 ciphers.
: OPENSSL_RET_NOT_OK(
: SSL_CTX_set_ciphersuites(ctx, tls_1_3_ciphers_.c_str()),
: Substitute("failed to set TLSv1.3 ciphers: $0", tls_1_3_ciphers_));
: #endif
I have one drive-by comment that I just noticed:
I think we want the SSL_CTX_set_ciphersuites() call to come before the SSL_CTX_set_cipher_list() call.
This revolves around whether the code will throw an error here if no ciphers are configured. SSL_CTX_set_ciphersuites() will throw an error if no ciphers are configured, but on some versions of OpenSSL (like 1.1.1 used on Ubuntu 18), it is including TLS 1.3 ciphersuites in the list. (This behavior is fixed in more recent OpenSSL.)
So, if tls_ciphers_ is "invalid_cipher" and tls_1_3_ciphers_ is the empty string (i.e. no TLS 1.3 ciphers), SSL_CTX_set_cipher_list() won't fail, because the TLS 1.3 ciphersuites default to having multiple ciphers available.
The SSL_CTX_set_ciphersuites() call doesn't actually check whether there are zero ciphers configured overall. So, it also wouldn't fail.
Reversing the calls, SSL_CTX_set_ciphersuites() with an empty string is valid and then the SSL_CTX_set_cipher_list() call will fail if there are truly no ciphers available (and it sees the accurate TLS 1.3 list).
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Fri, 26 Mar 2021 18:19:07 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed a vote on this change.
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 6:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17189/6//COMMIT_MSG
Commit Message:
PS6:
This changelist needs to be submitted along with the corresponding patch for the Java side. Otherwise, current Java Kudu clients would not be able to talk with servers built against OpenSSL 1.1.1 unless adding --rpc_tls_excluded_protocols=TLSv1.3 flag.
I'm planning to post the corresponding Java-side patch soon.
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 6
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Mon, 29 Mar 2021 22:07:12 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 8:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/security/security_flags.cc
File src/kudu/security/security_flags.cc:
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/security/security_flags.cc@29
PS8, Line 29: // TODO(aserbin): refresh the list to drop RHEL6/CentOS6 ciphers and
> Should that patch land first before this?
That's I'm sure about. We might want to keep backward compatibility for the client side for a couple of releases, phasing pre-TLSv1.2 in a couple of releases.
Basically, the idea is that it's possible to control the TLS protocol version and ciphers list on the server side, but it's not so for the client side. So, the safest path forward is to re-order and shorten the list, but not exactly removing pre-TLSv1.2 ciphers.
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/server/server_base.cc
File src/kudu/server/server_base.cc:
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/server/server_base.cc@167
PS8, Line 167: DEFINE_string(rpc_tls_1_3_ciphers,
> A flag tied to a version of a dependency feels like a future compatibility/
What's exactly the challenge here? Just consider this as the parameter to set TLSv1.3 and later version ciphers. Those are controlled by different call in OpenSSL API.
No, I don't think there much sense in re-using `rpc_tlc_ciphers` for TLSv1.3 ciphers: the syntax for the priority list and the API differs. And I don't think anybody wants to build a complicated and error-prone parser to separate TLSv1.3 and pre-TLSv1.3 ciphers from the list.
See https://www.openssl.org/docs/man1.1.1/man3/SSL_CTX_set_ciphersuites.html
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 8
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Mon, 05 Apr 2021 16:27:31 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Attila Bukor, Kudu Jenkins, Andrew Wong, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17189
to look at the new patch set (#4).
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
With this patch Kudu servers and Kudu C++ client can use TLSv1.3
for securing Kudu RPC communication. All tests run between C++
components now use TLSv1.3 when compiled with OpenSSL 1.1.1 or newer
versions.
This patch introduces two new server-side flags:
--rpc_tls_1_3_ciphers
--rpc_tls_excluded_protocols
The former is to customize the list of preferred ciphers for TLSv1.3.
The latter is to control the set of TLS protocols used by Kudu servers
when securing connections for RPC communication. It can be used along
with the --rpc_tls_min_protocol flag to define the set of TLS protocols
available to Kudu servers when running a TLS handshake. For example,
set --rpc_tls_excluded_protocols=TLSv1.3 to exclude TLSv1.3 from
available options for securing Kudu RPC.
In addition, I added a validator for the --rpc_tls_min_protocol flag,
similar to the validator for --rpc_tls_excluded_protocols.
This patch also adds a new test to exercise TLSv1.3 handshake in
the context of establishing a connection for Kudu RPC.
The embedded webserver isn't updated yet with TLSv1.3 cipher controls:
that will be done in a separate patch because it requires pushing an
updated to squeasel to introduce a new TLSv1.3-specific flag and call
SSL_CTX_set_ciphersuites() correspondingly.
I ran some of the existing performance tests to compare TLSv1.2
vs TLSv1.3 performance on 48-core machine with
Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz.
Parameters/TestRpc.TestPerformanceBySocketType/SSL_TCP
Sending 1024MB via ssl-enabled tcp socket:
TLSv1.2: real 17.774s user 28.992s sys 2.473s
TLSv1.3: real 2.702s user 1.461s sys 3.112s
RpcBench.BenchmarkCalls
TLSv1.2
Reqs/sec: 64364.7
User CPU per req: 65.4646us
Sys CPU per req: 84.3419us
Ctx Sw. per req: 4.02794
Server reactor latency histogram
Count: 1938009
Mean: 33.1368
TLSv1.3
Reqs/sec: 117503
User CPU per req: 27.9537us
Sys CPU per req: 79.2585us
Ctx Sw. per req: 3.50379
Server reactor latency histogram
Count: 3815425
Mean: 17.695
Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
---
M src/kudu/rpc/client_negotiation.cc
M src/kudu/rpc/messenger.cc
M src/kudu/rpc/messenger.h
M src/kudu/rpc/server_negotiation.cc
M src/kudu/security/security-test-util.cc
M src/kudu/security/security_flags.cc
M src/kudu/security/security_flags.h
M src/kudu/security/tls_context.cc
M src/kudu/security/tls_context.h
M src/kudu/security/tls_handshake-test.cc
M src/kudu/security/tls_handshake.cc
M src/kudu/security/tls_handshake.h
M src/kudu/server/server_base.cc
13 files changed, 468 insertions(+), 90 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/89/17189/4
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Attila Bukor, Kudu Jenkins, Andrew Wong, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17189
to look at the new patch set (#5).
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
With this patch Kudu servers and Kudu C++ client can use TLSv1.3
for securing Kudu RPC communication. All tests run between C++
components now use TLSv1.3 when compiled with OpenSSL 1.1.1 or newer
versions.
This patch introduces two new server-side flags:
--rpc_tls_1_3_ciphers
--rpc_tls_excluded_protocols
The former is to customize the list of preferred ciphers for TLSv1.3.
The latter is to control the set of TLS protocols used by Kudu servers
when securing connections for RPC communication. It can be used along
with the --rpc_tls_min_protocol flag to define the set of TLS protocols
available to Kudu servers when running a TLS handshake. For example,
set --rpc_tls_excluded_protocols=TLSv1.3 to exclude TLSv1.3 from
available options for securing Kudu RPC.
In addition, I added a validator for the --rpc_tls_min_protocol flag,
similar to the validator for --rpc_tls_excluded_protocols.
This patch also adds a new test to exercise TLSv1.3 handshake in
the context of establishing a connection for Kudu RPC.
The embedded webserver isn't updated yet with TLSv1.3 cipher controls:
that will be done in a separate patch because it requires pushing an
updated to squeasel to introduce a new TLSv1.3-specific flag and call
SSL_CTX_set_ciphersuites() correspondingly.
I ran some of the existing performance tests to compare TLSv1.2
vs TLSv1.3 performance on 48-core machine with
Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz.
Parameters/TestRpc.TestPerformanceBySocketType/SSL_TCP
Sending 1024MB via ssl-enabled tcp socket:
TLSv1.2: real 17.774s user 28.992s sys 2.473s
TLSv1.3: real 2.702s user 1.461s sys 3.112s
RpcBench.BenchmarkCalls
TLSv1.2
Reqs/sec: 64364.7
User CPU per req: 65.4646us
Sys CPU per req: 84.3419us
Ctx Sw. per req: 4.02794
Server reactor latency histogram
Count: 1938009
Mean: 33.1368
TLSv1.3
Reqs/sec: 117503
User CPU per req: 27.9537us
Sys CPU per req: 79.2585us
Ctx Sw. per req: 3.50379
Server reactor latency histogram
Count: 3815425
Mean: 17.695
Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
---
M src/kudu/rpc/client_negotiation.cc
M src/kudu/rpc/messenger.cc
M src/kudu/rpc/messenger.h
M src/kudu/rpc/server_negotiation.cc
M src/kudu/security/security-test-util.cc
M src/kudu/security/security_flags.cc
M src/kudu/security/security_flags.h
M src/kudu/security/tls_context.cc
M src/kudu/security/tls_context.h
M src/kudu/security/tls_handshake-test.cc
M src/kudu/security/tls_handshake.cc
M src/kudu/security/tls_handshake.h
M src/kudu/server/server_base.cc
13 files changed, 470 insertions(+), 92 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/89/17189/5
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 5: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 23 Mar 2021 18:40:43 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has removed a vote on this change.
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Removed Verified-1 by Kudu Jenkins (120)
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: deleteVote
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 10
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 8:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/security/security_flags.cc
File src/kudu/security/security_flags.cc:
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/security/security_flags.cc@29
PS8, Line 29: // TODO(aserbin): refresh the list to drop RHEL6/CentOS6 ciphers and
> That's I'm sure about. We might want to keep backward compatibility for th
Makes sense.
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/server/server_base.cc
File src/kudu/server/server_base.cc:
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/server/server_base.cc@167
PS8, Line 167: DEFINE_string(rpc_tls_1_3_ciphers,
> What's exactly the challenge here? Just consider this as the parameter to
The OpenSSL API doesn't use the version number to signify the difference instead it uses ciphersuites vs cipherlist. Maybe to match the OpenSSL terminoligy this flag should be rpc_tls_ciphersuites with similar documentation to OpenSSL on the difference?
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 8
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 06 Apr 2021 14:23:23 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 2:
(9 comments)
http://gerrit.cloudera.org:8080/#/c/17189/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17189/2//COMMIT_MSG@33
PS2, Line 33: requires first
: to squeasel
nit: "requires pushing an update to squeasel"?
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/rpc/messenger.cc
File src/kudu/rpc/messenger.cc:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/rpc/messenger.cc@148
PS2, Line 148: std::string
nit: can drop the std::s here and elsewhere
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/rpc/server_negotiation.cc
File src/kudu/rpc/server_negotiation.cc:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/rpc/server_negotiation.cc@610
PS2, Line 610: // Check that the handshake step didn't produce an error. It also propagates
: // any non-OK status.
: RETURN_NOT_OK(s);
Wouldn't this return early if 's' is Incomplete in the needs_extra_step case, even if we've already handled the needs_extra_step above? Should we reset 's' after L607?
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.h
File src/kudu/security/tls_context.h:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.h@181
PS2, Line 181: // The cipher suite preferences to use for TLS-secured RPC connections. Uses the OpenSSL
: // cipher preference list format. See man (1) ciphers for more information.
nit: I suppose it's implied from the below comment, but maybe also mention here that these are pre-1.3 ciphers only?
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.cc
File src/kudu/security/tls_context.cc:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.cc@117
PS2, Line 117: constexpr const char* const kTLSv1_1 = "TLSv1.1";
> kKTlSv11 instead of kTLSv1_1 ? No, thank you :)
+11 :)
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.cc@241
PS2, Line 241:
nit: add space
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_handshake-test.cc
File src/kudu/security/tls_handshake-test.cc:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_handshake-test.cc@327
PS2, Line 327: server.set_verification_mode(security::TlsVerificationMode::VERIFY_NONE);
nit: mind adding a comment whether/why this is important for this test?
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_handshake-test.cc@358
PS2, Line 358: verfy
nit: very
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_handshake-test.cc@375
PS2, Line 375:
nit: extra space
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Mon, 22 Mar 2021 22:13:02 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 6:
(1 comment)
The updated tls_context.cc looks good to me.
http://gerrit.cloudera.org:8080/#/c/17189/5/src/kudu/security/tls_context.cc
File src/kudu/security/tls_context.cc:
http://gerrit.cloudera.org:8080/#/c/17189/5/src/kudu/security/tls_context.cc@259
PS5, Line 259: // those might be none. From the other side, the implementation of
: // SSL_CTX_set_cipher_list() verifies that the overall result list of ciphers
: // is valid and usable, reporting an error otherwise.
: //
: // However, if the sequence is reversed, no error would be reported in case
: // o
> Thank you for the feedback!
Thanks for fixing this!
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 6
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 30 Mar 2021 00:11:19 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 8:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/server/server_base.cc
File src/kudu/server/server_base.cc:
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/server/server_base.cc@167
PS8, Line 167: DEFINE_string(rpc_tls_1_3_ciphers,
> Yep, let's explore this route. With '--rpc_tls_ciphersuites' my concern is
I agree the naming is close. That said, anyone who knows openssl enough to know to set the cyphers is also probably familiar with the naming convention set by openssl. The naming scheme is uniform across openssl and kudu.
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 8
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 06 Apr 2021 17:44:35 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Attila Bukor, Kudu Jenkins, Andrew Wong, Grant Henke, Joe McDonnell,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17189
to look at the new patch set (#9).
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
With this patch Kudu servers and Kudu C++ client can use TLSv1.3
for securing Kudu RPC communication. All tests run between C++
components now use TLSv1.3 when compiled with OpenSSL 1.1.1 or newer.
This patch introduces two new server-side flags:
--rpc_tls_ciphersuites
--rpc_tls_excluded_protocols
The former is to customize the list of preferred ciphers for TLSv1.3.
The latter is to control the set of TLS protocols used by Kudu servers
when securing connections for RPC communication. It can be used along
with the --rpc_tls_min_protocol flag to define the set of TLS protocols
available to Kudu servers when running a TLS handshake. For example,
set --rpc_tls_excluded_protocols=TLSv1.3 to exclude TLSv1.3 from
available options for securing Kudu RPC.
In addition, I added a validator for the --rpc_tls_min_protocol flag,
similar to the validator for --rpc_tls_excluded_protocols.
This patch also adds a new test to exercise TLSv1.3 handshake in
the context of establishing a connection for Kudu RPC.
The embedded webserver isn't updated yet with TLSv1.3 cipher controls.
That will be done in a separate patch because it requires pushing an
update to the squeasel webserver to introduce a new TLSv1.3-specific
flag and make a call to SSL_CTX_set_ciphersuites() correspondingly.
Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
---
M src/kudu/rpc/client_negotiation.cc
M src/kudu/rpc/messenger.cc
M src/kudu/rpc/messenger.h
M src/kudu/rpc/server_negotiation.cc
M src/kudu/security/security-test-util.cc
M src/kudu/security/security_flags.cc
M src/kudu/security/security_flags.h
M src/kudu/security/tls_context.cc
M src/kudu/security/tls_context.h
M src/kudu/security/tls_handshake-test.cc
M src/kudu/security/tls_handshake.cc
M src/kudu/security/tls_handshake.h
M src/kudu/server/server_base.cc
13 files changed, 520 insertions(+), 99 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/89/17189/9
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 9
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 8:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/security/security_flags.cc
File src/kudu/security/security_flags.cc:
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/security/security_flags.cc@29
PS8, Line 29: // TODO(aserbin): refresh the list to drop RHEL6/CentOS6 ciphers and
Should that patch land first before this?
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/server/server_base.cc
File src/kudu/server/server_base.cc:
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/server/server_base.cc@167
PS8, Line 167: DEFINE_string(rpc_tls_1_3_ciphers,
A flag tied to a version of a dependency feels like a future compatibility/usability challenge to me. Will this flag also be used for 1.4 or later?
Is there a way that the `rpc_tls_ciphers` can be updated/reused in a compatible way? Perhaps filtering out incompatible cyphers or changing the default value based on the availability of OpenSSL v1.1.1 at compile time.
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 8
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Mon, 05 Apr 2021 14:34:32 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 2: Verified+1
unrelated test failure in MasterReplicationAndRpcSizeLimitTest.TabletReports
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Fri, 19 Mar 2021 19:05:19 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 4: Code-Review+2
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/rpc/server_negotiation.cc
File src/kudu/rpc/server_negotiation.cc:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/rpc/server_negotiation.cc@610
PS2, Line 610: // Check that the handshake step didn't produce an error. It also propagates
: // any non-OK status.
: RETURN_NOT_OK(s);
> It's designed to return early from here if 's' is non-OK, both for Status::
Ah indeed, and seems that was the behavior before as well. My mistake -- I thought it wasn't the case before.
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 4
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 23 Mar 2021 17:50:31 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Attila Bukor, Kudu Jenkins, Andrew Wong, Grant Henke, Joe McDonnell,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17189
to look at the new patch set (#10).
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
With this patch Kudu servers and Kudu C++ client can use TLSv1.3
for securing Kudu RPC communication. All tests run between C++
components now use TLSv1.3 when compiled with OpenSSL 1.1.1 or newer.
This patch introduces two new server-side flags:
--rpc_tls_ciphersuites
--rpc_tls_excluded_protocols
The former is to customize the list of preferred ciphers for TLSv1.3.
The latter is to control the set of TLS protocols used by Kudu servers
when securing connections for RPC communication. It can be used along
with the --rpc_tls_min_protocol flag to define the set of TLS protocols
available to Kudu servers when running a TLS handshake. For example,
set --rpc_tls_excluded_protocols=TLSv1.3 to exclude TLSv1.3 from
available options for securing Kudu RPC.
In addition, I added a validator for the --rpc_tls_min_protocol flag,
similar to the validator for --rpc_tls_excluded_protocols.
This patch also adds a new test to exercise TLSv1.3 handshake in
the context of establishing a connection for Kudu RPC.
The embedded webserver isn't updated yet with TLSv1.3 cipher controls.
That will be done in a separate patch because it requires pushing an
update to the squeasel webserver to introduce a new TLSv1.3-specific
flag and make a call to SSL_CTX_set_ciphersuites() correspondingly.
Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
---
M src/kudu/rpc/client_negotiation.cc
M src/kudu/rpc/messenger.cc
M src/kudu/rpc/messenger.h
M src/kudu/rpc/server_negotiation.cc
M src/kudu/security/security-test-util.cc
M src/kudu/security/security_flags.cc
M src/kudu/security/security_flags.h
M src/kudu/security/tls_context.cc
M src/kudu/security/tls_context.h
M src/kudu/security/tls_handshake-test.cc
M src/kudu/security/tls_handshake.cc
M src/kudu/security/tls_handshake.h
M src/kudu/server/server_base.cc
13 files changed, 529 insertions(+), 99 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/89/17189/10
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 10
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 5:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17189/5/src/kudu/security/tls_context.cc
File src/kudu/security/tls_context.cc:
http://gerrit.cloudera.org:8080/#/c/17189/5/src/kudu/security/tls_context.cc@259
PS5, Line 259: #if OPENSSL_VERSION_NUMBER >= 0x10101000L
: // Set TLSv1.3 ciphers.
: OPENSSL_RET_NOT_OK(
: SSL_CTX_set_ciphersuites(ctx, tls_1_3_ciphers_.c_str()),
: Substitute("failed to set TLSv1.3 ciphers: $0", tls_1_3_ciphers_));
: #endif
> I have one drive-by comment that I just noticed:
Thank you for the feedback!
I added a blurb about this into the code.
I also verified that in case of OpenSSL 1.1.1i and OpenSSL 1.1.1j regardless of the sequence of those calls TLS context reports an error. E.g., kudu-master isn't starting if setting --rpc_tls_ciphers="" --rpc_tls_1_3_ciphers="" with the following error in both cases:
E0329 11:10:11.678535 150270 master_main.cc:40] Runtime error: RunMasterServer() failed: failed to set TLS ciphers: : error:1410D0B9:SSL routines:SSL_CTX_set_cipher_list:no cipher match:../ssl/ssl_lib.c:2564
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 5
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Mon, 29 Mar 2021 19:42:11 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Andrew Wong (Code Review)" <ge...@cloudera.org>.
Andrew Wong has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 6: Code-Review+2
LGTM, but would appreciate another look from Joe on the most recent blurb (and happy to wait on the Java patch as well).
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 6
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 30 Mar 2021 00:04:44 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Attila Bukor, Kudu Jenkins, Andrew Wong, Grant Henke, Joe McDonnell,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17189
to look at the new patch set (#6).
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
With this patch Kudu servers and Kudu C++ client can use TLSv1.3
for securing Kudu RPC communication. All tests run between C++
components now use TLSv1.3 when compiled with OpenSSL 1.1.1 or newer
versions.
This patch introduces two new server-side flags:
--rpc_tls_1_3_ciphers
--rpc_tls_excluded_protocols
The former is to customize the list of preferred ciphers for TLSv1.3.
The latter is to control the set of TLS protocols used by Kudu servers
when securing connections for RPC communication. It can be used along
with the --rpc_tls_min_protocol flag to define the set of TLS protocols
available to Kudu servers when running a TLS handshake. For example,
set --rpc_tls_excluded_protocols=TLSv1.3 to exclude TLSv1.3 from
available options for securing Kudu RPC.
In addition, I added a validator for the --rpc_tls_min_protocol flag,
similar to the validator for --rpc_tls_excluded_protocols.
This patch also adds a new test to exercise TLSv1.3 handshake in
the context of establishing a connection for Kudu RPC.
The embedded webserver isn't updated yet with TLSv1.3 cipher controls:
that will be done in a separate patch because it requires pushing an
update to squeasel to introduce a new TLSv1.3-specific flag and call
SSL_CTX_set_ciphersuites() correspondingly.
I ran some of the existing performance tests to compare TLSv1.2
vs TLSv1.3 performance on 48-core machine with
Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz.
Parameters/TestRpc.TestPerformanceBySocketType/SSL_TCP
Sending 1024MB via ssl-enabled tcp socket:
TLSv1.2: real 17.774s user 28.992s sys 2.473s
TLSv1.3: real 2.702s user 1.461s sys 3.112s
RpcBench.BenchmarkCalls
TLSv1.2
Reqs/sec: 64364.7
User CPU per req: 65.4646us
Sys CPU per req: 84.3419us
Ctx Sw. per req: 4.02794
Server reactor latency histogram
Count: 1938009
Mean: 33.1368
TLSv1.3
Reqs/sec: 117503
User CPU per req: 27.9537us
Sys CPU per req: 79.2585us
Ctx Sw. per req: 3.50379
Server reactor latency histogram
Count: 3815425
Mean: 17.695
Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
---
M src/kudu/rpc/client_negotiation.cc
M src/kudu/rpc/messenger.cc
M src/kudu/rpc/messenger.h
M src/kudu/rpc/server_negotiation.cc
M src/kudu/security/security-test-util.cc
M src/kudu/security/security_flags.cc
M src/kudu/security/security_flags.h
M src/kudu/security/tls_context.cc
M src/kudu/security/tls_context.h
M src/kudu/security/tls_handshake-test.cc
M src/kudu/security/tls_handshake.cc
M src/kudu/security/tls_handshake.h
M src/kudu/server/server_base.cc
13 files changed, 486 insertions(+), 92 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/89/17189/6
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 6
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Hello Tidy Bot, Attila Bukor, Kudu Jenkins, Andrew Wong, Grant Henke,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/17189
to look at the new patch set (#3).
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
With this patch Kudu servers and Kudu C++ client can use TLSv1.3
for securing Kudu RPC communication. All tests run between C++
components now use TLSv1.3 when compiled with OpenSSL 1.1.1 or newer
versions.
This patch introduces two new server-side flags:
--rpc_tls_1_3_ciphers
--rpc_tls_excluded_protocols
The former is to customize the list of preferred ciphers for TLSv1.3.
The latter is to control the set of TLS protocols used by Kudu servers
when securing connections for RPC communication. It can be used along
with the --rpc_tls_min_protocol flag to define the set of TLS protocols
available to Kudu servers when running a TLS handshake. For example,
set --rpc_tls_excluded_protocols=TLSv1.3 to exclude TLSv1.3 from
available options for securing Kudu RPC.
In addition, I added a validator for the --rpc_tls_min_protocol flag,
similar to the validator for --rpc_tls_excluded_protocols.
This patch also adds a new test to exercise TLSv1.3 handshake in
the context of establishing a connection for Kudu RPC.
The embedded webserver isn't updated yet with TLSv1.3 cipher controls:
that will be done in a separate patch because it requires first
to squeasel to introduce a new TLSv1.3-specific flag and call
SSL_CTX_set_ciphersuites() correspondingly.
I ran some of the existing performance tests to compare TLSv1.2
vs TLSv1.3 performance on 48-core machine with
Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50GHz.
Parameters/TestRpc.TestPerformanceBySocketType/SSL_TCP
Sending 1024MB via ssl-enabled tcp socket:
TLSv1.2: real 17.774s user 28.992s sys 2.473s
TLSv1.3: real 2.702s user 1.461s sys 3.112s
RpcBench.BenchmarkCalls
TLSv1.2
Reqs/sec: 64364.7
User CPU per req: 65.4646us
Sys CPU per req: 84.3419us
Ctx Sw. per req: 4.02794
Server reactor latency histogram
Count: 1938009
Mean: 33.1368
TLSv1.3
Reqs/sec: 117503
User CPU per req: 27.9537us
Sys CPU per req: 79.2585us
Ctx Sw. per req: 3.50379
Server reactor latency histogram
Count: 3815425
Mean: 17.695
Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
---
M src/kudu/rpc/client_negotiation.cc
M src/kudu/rpc/messenger.cc
M src/kudu/rpc/messenger.h
M src/kudu/rpc/server_negotiation.cc
M src/kudu/security/security-test-util.cc
M src/kudu/security/security_flags.cc
M src/kudu/security/security_flags.h
M src/kudu/security/tls_context.cc
M src/kudu/security/tls_context.h
M src/kudu/security/tls_handshake-test.cc
M src/kudu/security/tls_handshake.cc
M src/kudu/security/tls_handshake.h
M src/kudu/server/server_base.cc
13 files changed, 468 insertions(+), 90 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/89/17189/3
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 8:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/security/security_flags.cc
File src/kudu/security/security_flags.cc:
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/security/security_flags.cc@29
PS8, Line 29: // TODO(aserbin): refresh the list to drop RHEL6/CentOS6 ciphers and
> Makes sense.
Thank you for the feedback! I'll iterate on this idea, posting next revision for https://gerrit.cloudera.org/#/c/17268/
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/server/server_base.cc
File src/kudu/server/server_base.cc:
http://gerrit.cloudera.org:8080/#/c/17189/8/src/kudu/server/server_base.cc@167
PS8, Line 167: DEFINE_string(rpc_tls_1_3_ciphers,
> The OpenSSL API doesn't use the version number to signify the difference in
Yep, let's explore this route. With '--rpc_tls_ciphersuites' my concern is that '--rpc_tls_ciphersuites' very close to '--rpc_tls_ciphers', so it might be confusing to humans.
Anyways, I think we can go forward with this `--rpc_tls_ciphersuites`, adding corresponding description. I'm going to post a new revision shortly.
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 8
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 06 Apr 2021 17:15:35 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 2:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/rpc/server_negotiation.cc
File src/kudu/rpc/server_negotiation.cc:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/rpc/server_negotiation.cc@614
PS2, Line 614: if (!needs_extra_step && !token.empty()) {
> warning: 'token' used after it was moved [bugprone-use-after-move]
This is bogus: 'token' was not moved if 'needs_extra_step' evaluates to 'false'.
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.cc
File src/kudu/security/tls_context.cc:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.cc@117
PS2, Line 117: constexpr const char* const kTLSv1_1 = "TLSv1.1";
> warning: invalid case style for global constant 'kTLSv1_1' [readability-ide
kKTlSv11 instead of kTLSv1_1 ? No, thank you :)
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.cc@118
PS2, Line 118: constexpr const char* const kTLSv1_2 = "TLSv1.2";
> warning: invalid case style for global constant 'kTLSv1_2' [readability-ide
ditto
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.cc@119
PS2, Line 119: constexpr const char* const kTLSv1_3 = "TLSv1.3";
> warning: invalid case style for global constant 'kTLSv1_3' [readability-ide
ditto
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Fri, 19 Mar 2021 18:42:13 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Grant Henke (Code Review)" <ge...@cloudera.org>.
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 10: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 10
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Wed, 07 Apr 2021 12:42:34 +0000
Gerrit-HasComments: No
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17189/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17189/2//COMMIT_MSG@7
PS2, Line 7: (C++ part)
I'm planning to have a separate patch for the Java part: that's for the ease of reviewing and tracking TLSv1.3-related changes.
When pushing to the upstream git repo, this patch needs to submitted along with its Java counterpart, otherwise Java client would not be able to invoke RPCs against TLSv1.3-enabled Kudu servers.
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Sat, 20 Mar 2021 00:43:20 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 2:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/17189/2//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/17189/2//COMMIT_MSG@33
PS2, Line 33: requires first
: to squeasel
> nit: "requires pushing an update to squeasel"?
Done
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 2
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 23 Mar 2021 03:46:26 +0000
Gerrit-HasComments: Yes
[kudu-CR] KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
Posted by "Alexey Serbin (Code Review)" <ge...@cloudera.org>.
Alexey Serbin has posted comments on this change. ( http://gerrit.cloudera.org:8080/17189 )
Change subject: KUDU-2871 support TLSv1.3 in Kudu RPC (C++ part)
......................................................................
Patch Set 3:
(7 comments)
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/rpc/messenger.cc
File src/kudu/rpc/messenger.cc:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/rpc/messenger.cc@148
PS2, Line 148: nal-facing
> nit: can drop the std::s here and elsewhere
I posted a separate patch to address these nits in messenger.{h,cc}
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/rpc/server_negotiation.cc
File src/kudu/rpc/server_negotiation.cc:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/rpc/server_negotiation.cc@610
PS2, Line 610: // Check that the handshake step didn't produce an error. It also propagates
: // any non-OK status.
: RETURN_NOT_OK(s);
> Wouldn't this return early if 's' is Incomplete in the needs_extra_step cas
It's designed to return early from here if 's' is non-OK, both for Status::Incomplete() and other non-OK cases. At line 607 we just send back the information produced by TLS handshake, but the essence of the HandleTlsHandshake() method is to convey status from TlsHandshake::Continue() to the upper level.
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.h
File src/kudu/security/tls_context.h:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.h@181
PS2, Line 181: // The cipher suite preferences to use for RPC connections secured with
: // pre-TLSv1.3 protocols. Uses the OpenSSL cipher preference list format.
> nit: I suppose it's implied from the below comment, but maybe also mention
Done
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.cc
File src/kudu/security/tls_context.cc:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_context.cc@241
PS2, Line 241:
> nit: add space
Done
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_handshake-test.cc
File src/kudu/security/tls_handshake-test.cc:
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_handshake-test.cc@327
PS2, Line 327: // The client has a self-signed certificate, so the server has nothing to
> nit: mind adding a comment whether/why this is important for this test?
Done
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_handshake-test.cc@358
PS2, Line 358:
> nit: very
Done
http://gerrit.cloudera.org:8080/#/c/17189/2/src/kudu/security/tls_handshake-test.cc@375
PS2, Line 375: a
> nit: extra space
Done
--
To view, visit http://gerrit.cloudera.org:8080/17189
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia92a4d102c3c8cff76101e71ff71d24a9d78b672
Gerrit-Change-Number: 17189
Gerrit-PatchSet: 3
Gerrit-Owner: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Alexey Serbin <as...@cloudera.com>
Gerrit-Reviewer: Andrew Wong <aw...@cloudera.com>
Gerrit-Reviewer: Attila Bukor <ab...@apache.org>
Gerrit-Reviewer: Grant Henke <gr...@apache.org>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Comment-Date: Tue, 23 Mar 2021 03:33:46 +0000
Gerrit-HasComments: Yes