[jira] [Commented] (KUDU-2905) Impala queries failed when master's IPKI CA info changed

["Alexey Serbin (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KUDU-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17736301#comment-17736301 ] 

Alexey Serbin commented on KUDU-2905:
-------------------------------------

The IPKI CA certificate of a Kudu cluster isn't supposed to change.

An idea to automatically refresh Kudu cluster's certificate that is already know to a client seems quite unsafe from the security standpoint.  If a Kudu client detects that the certificate of the cluster it's been working has changed, it might be an attempt to hijack a communication channel between the cluster and the client.  If a Kudu client has acquired an authentication token and uses it for authentication along with CA certificate, it's no longer tries to use the primary credentials for authentication (such as Kerberos creds).  With that, it has no other way of authenticating the servers it talks to except for verifying TLS certificates of the servers it talks to against the CA certificate it has on hand.

So, if a cluster's IPKI certificate has changed, that should be considered as an exceptional situation, and I don't see any way to automate handling this while not compromising security of Kudu RPC.  Should Kudu cluster's certificate change, all runing client applications should be restarted.

> Impala queries failed when master's IPKI CA info changed
> --------------------------------------------------------
>
>                 Key: KUDU-2905
>                 URL: https://issues.apache.org/jira/browse/KUDU-2905
>             Project: Kudu
>          Issue Type: Bug
>          Components: client, master, security
>    Affects Versions: 1.11.0
>            Reporter: Adar Dembo
>            Priority: Major
>
> Saw this in a user report.
> The cluster in question lost its master and the state was rebuilt from that of the tservers (see KUDU-2902). In doing so, the master lost its IPKI cert info and a new record was generated.
> After the Kudu cluster was operational, the existing Impala cluster could not issue queries. All queries failed with an error like this:
> {noformat}
> Unable to open Kudu table: Runtime error: Client connection negotiation failed: client connection to 10.38.202.4:7051: TLS Handshake error: error:04067084:rsa routines:RSA_EAY_PUBLIC_DECRYPT:data too large for modulus:rsa_eay.c:738 error:0D0C5006:asn1 encoding routines:ASN1_item_verify:EVP lib:a_verify.c:249 error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed:s3_clnt.c:1264
> {noformat}
> Restarting Impala fixed it.
> I'm not sure if this is an issue with how Impala caches KuduClient instances, or if it's an issue with how the client itself caches the master's CA certificate. For now I'm assuming this is a Kudu issue and the client needs to detect this error and invalidate existing certificates.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)