RE: Need help for TLS implementation with CA signed certificates

[Krishnakishore Ganta <kr...@ica.se>]

Hi All,

I am  implementing TLS for  NIFI and we are able to connect to primary node with https and nifi page is displaying cluster with 1/3 status. We observed following errors in nifi-app.log file -

2019-12-23 14:01:47,286 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed to create socket to node03:9081 due to: java.net.ConnectException: Connection refused (Connection refused)
2019-12-23 14:01:52,288 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at node03:9081; will use this address for sending heartbeat messages
2019-12-23 14:01:52,367 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
2019-12-23 14:01:57,371 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at node03:9081; will use this address for sending heartbeat messages
2019-12-23 14:01:57,392 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
2019-12-23 14:02:02,395 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at node03:9081; will use this address for sending heartbeat messages
2019-12-23 14:02:02,409 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown

I have few questions before fixing above errors, please correct if I am doing wrong with below points.

Just and update before jumping into my questions : I am using my company signed CA certificate (which varies from server to server)  not intermediate CA certificate which is common across organization.


  1.  When using nifi toolkit there are 2 ways of generating certificates – Standalone mode and  Server/client mode.

  1.  Standalone mode : To generate following files with single  command :  nifi-cert.pem,nifi-key.key,CN=username_OU=ou.password,CN=username_OU=ou.p12 and xyz directory, where xyz directory again contains - keystore.jks, nifi.properties and truststore.jks :  My question here is -  Do we need to modify anything in nifi.properties file here ?
  2.  Server/ client Mode : This is used to make one of the server as certificate approving authority and other nodes as clients. Since we are already using organization signed CA certificate, I am going with Standalone mode.

  1.  Modified nifi.properties file which is present in nifi conf directory to read new ports after TLS, keystore, truststore and password.
  2.  Imported our organization rootca and intermediate ca files into both keystore.jks and truststore.jks along with other nodes signed certificates mentioned in point 1.
  3.  As part of troubleshooting  to fix above error, added/imported  nifi-cert.pem which is generated by nifi toolkit mentioned in point 2 to other nodes keystore.jks and truststore.jks, even this didn’t fix my errors.
  4.  Apart from modifying nifi.properties file mentioned in point 3, do we need to any other files? Like - authorizations.xml, authorizers.xml, users.xml etc , if yes – please suggestion on what and how to modify.

Please guide me on above mentioned/asked questions or correct me if I am going with wrong assumptions. I tried with my understanding 😊

Thank you in advance for taking your time in answering. Waiting for response.

Thanks
Krishna

________________________________
ATTENTION: This e-mail may contain confidential information that is intended solely for the addressee. If you are not the intended recipient, you should delete this message and are hereby notified that any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited.
________________________________
PRIVACY NOTICE: Your privacy is important for us at ICA Gruppen AB and its subsidiaries (ICA). We are transparent with how we collect and process any personal data that you share with us. More detailed information on how we process your personal data can be found at www.ica.se/dataskydd.

Re: Need help for TLS implementation with CA signed certificates

[Andy LoPresto <al...@apache.org>]

You will likely want to read the TLS Toolkit Guide [1] closely, especially the section for using an external CA [2] or externally-signed certificates [3]. Whether generated using the TLS Toolkit or provided from an external source (i.e. your organization generates signed certificates per NiFi node), the truststore for each node needs to be aware of the presented public certificates of every other node. Therefore, the easiest solution is to use a common intermediate CA to sign all node certificates and import the public certificate of the CA into a single truststore which is present on every node. However, you can use external certificates, provided that either each public certificate is populated in every truststore, or a common ancestor of all node certificates is. 

When you generate the certificates using TLS Toolkit, the resulting files in the output directories each contain the keystore, truststore, and generated nifi.properties file for a single node. The generated nifi.properties file has been populated with the keystore and truststore locations and passwords, and that file is based either on a generic template or the existing nifi.properties file being used by the node, depending on the command-line flags provided. If it uses the actual nifi.properties file, you can copy this newly-generated file directly into the conf/ directory. However, if you use the generic template or there are additional changes you need to make (for example, you generate the certificates one at a time and don’t configure the cluster settings), you’ll have to merge these changes manually. 

Importing nifi-cert.pem (the public certificate of the NiFi CA) into the truststore for each node is already part of the toolkit process. However, if you run the toolkit command independently on each node, it will generate a unique CA certificate on each node, and you will have to cross-import these CA certs into every truststore. Again, the recommended process is to generate all of the certs at once in the same location, thus using the same CA cert to sign all the certificates, or use the client/server mode to generate a single CA cert in one node and use it to sign all other certificates.  

[1] https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#tls_toolkit <https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#tls_toolkit>
[2] https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#tls_intermediate_ca
[3] https://nifi.apache.org/docs/nifi-docs/html/toolkit-guide.html#tls_external-signed_ca


Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69

> On Dec 24, 2019, at 8:02 AM, Krishnakishore Ganta <kr...@ica.se> wrote:
> 
> Hi All, 
>  
> I am  implementing TLS for  NIFI and we are able to connect to primary node with https and nifi page is displaying cluster with 1/3 status. We observed following errors in nifi-app.log file -
>  
> 2019-12-23 14:01:47,286 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed to create socket to node03:9081 due to: java.net.ConnectException: Connection refused (Connection refused)
> 2019-12-23 14:01:52,288 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at node03:9081; will use this address for sending heartbeat messages
> 2019-12-23 14:01:52,367 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
> 2019-12-23 14:01:57,371 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at node03:9081; will use this address for sending heartbeat messages
> 2019-12-23 14:01:57,392 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
> 2019-12-23 14:02:02,395 INFO [main] o.a.n.c.c.n.LeaderElectionNodeProtocolSender Determined that Cluster Coordinator is located at node03:9081; will use this address for sending heartbeat messages
> 2019-12-23 14:02:02,409 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: Failed marshalling 'CONNECTION_REQUEST' protocol message due to: javax.net.ssl.SSLHandshakeException: Received fatal alert: certificate_unknown
>  
> I have few questions before fixing above errors, please correct if I am doing wrong with below points.
>  
> Just and update before jumping into my questions : I am using my company signed CA certificate (which varies from server to server)  not intermediate CA certificate which is common across organization.
>  
> When using nifi toolkit there are 2 ways of generating certificates – Standalone mode and  Server/client mode.
> Standalone mode : To generate following files with single  command :  nifi-cert.pem,nifi-key.key,CN=username_OU=ou.password,CN=username_OU=ou.p12 and xyz directory, where xyz directory again contains - keystore.jks, nifi.properties and truststore.jks :  My question here is -  Do we need to modify anything in nifi.properties file here ?
> Server/ client Mode : This is used to make one of the server as certificate approving authority and other nodes as clients. Since we are already using organization signed CA certificate, I am going with Standalone mode.
> Modified nifi.properties file which is present in nifi conf directory to read new ports after TLS, keystore, truststore and password.
> Imported our organization rootca and intermediate ca files into both keystore.jks and truststore.jks along with other nodes signed certificates mentioned in point 1.
> As part of troubleshooting  to fix above error, added/imported  nifi-cert.pem which is generated by nifi toolkit mentioned in point 2 to other nodes keystore.jks and truststore.jks, even this didn’t fix my errors. 
> Apart from modifying nifi.properties file mentioned in point 3, do we need to any other files? Like - authorizations.xml, authorizers.xml, users.xml etc , if yes – please suggestion on what and how to modify. 
>  
> Please guide me on above mentioned/asked questions or correct me if I am going with wrong assumptions. I tried with my understanding 😊
>  
> Thank you in advance for taking your time in answering. Waiting for response.
>  
> Thanks
> Krishna
>  
> ATTENTION: This e-mail may contain confidential information that is intended solely for the addressee. If you are not the intended recipient, you should delete this message and are hereby notified that any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited.
> PRIVACY NOTICE: Your privacy is important for us at ICA Gruppen AB and its subsidiaries (ICA). We are transparent with how we collect and process any personal data that you share with us. More detailed information on how we process your personal data can be found at www.ica.se/dataskydd <http://www.ica.se/dataskydd>.