You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Conrad Crampton <co...@SecData.com> on 2016/02/15 17:39:05 UTC
SSL configuration on cluster
Hi,
New to this list and loving NiFi so far ;-)
I have set up and been trying out with good results a NiFi set up locally on my laptop – all good. Managed to set up SSL configuration and hook up with company LDAP.
I have also set up three node cluster with cluster manager on a separate host, all using http – all good here too. NCM communicates fine with three nodes, and vice versa. Ok, moving on to set up cluster to use SSL and LDAP is where I have come a bit unstuck.
I am using self signed certificate – on laptop used one generated using openssl, then converted to pks file (which worked fine as above).
The issue I’m having is running virtually the same setup on cluster as laptop fails with the following exception (there is loads of other nested exception details) but it appears to boil down to this one.
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'protocolSocketConfiguration': FactoryBean threw exception on object creation; nested exception is java.security.KeyStoreException: not found
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175) ~[na:na]
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) ~[na:na]
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1517) ~[na:na]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:314) ~[na:na]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194) ~[na:na]
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351) ~[na:na]
... 103 common frames omitted
Caused by: java.security.KeyStoreException: not found
at java.security.KeyStore.getInstance(KeyStore.java:851) ~[na:1.8.0_51]
at org.apache.nifi.io.socket.SSLContextFactory.<init>(SSLContextFactory.java:72) ~[na:na]
at org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:46) ~[na:na]
at org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:31) ~[na:na]
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) ~[na:na]
... 108 common frames omitted
Caused by: java.security.NoSuchAlgorithmException: KeyStore not available
at sun.security.jca.GetInstance.getInstance(GetInstance.java:159) ~[na:1.8.0_51]
at java.security.Security.getImpl(Security.java:695) ~[na:1.8.0_51]
at java.security.KeyStore.getInstance(KeyStore.java:848) ~[na:1.8.0_51]
Having looked at the code, line 72 (&73) of SSLContextFactory is
final KeyStore trustStore = KeyStore.getInstance(truststoreType);
final FileInputStream trustStoreStream = new FileInputStream(truststore);
Which is throwing the error by the looks of it, so appears to need a value in the nifi.properties file for nifi.security.truststoreType= which on my laptop config is blank (docs seems to suggest that this is optional anyway), but the code complains about this not being present. If I enter a value (JKS or PKCS12) in here I get a filenotfound exception on line 73 instead as I haven’t provided a value for truststore location. So, do I need to provide truststore configuration in a clustered environment? If so, docs would helpful to point this out, if not anyone suggest something to fix this please? It took me 10 mins to set up on laptop, now after 6 hours of trying on cluster I’m stumped.
Thanks
Conrad
SecureData, combating cyber threats
______________________________________________________________________
The information contained in this message or any of its attachments may be privileged and confidential and intended for the exclusive use of the intended recipient. If you are not the intended recipient any disclosure, reproduction, distribution or other dissemination or use of this communications is strictly prohibited. The views expressed in this email are those of the individual and not necessarily of SecureData Europe Ltd. Any prices quoted are only valid if followed up by a formal written quote.
SecureData Europe Limited. Registered in England & Wales 04365896. Registered Address: SecureData House, Hermitage Court, Hermitage Lane, Maidstone, Kent, ME16 9NT
Re: SSL configuration on cluster
Posted by Matthew Clarke <ma...@gmail.com>.
Now that you have all the keys you need for the systems in your cluster,
you will need to create some keys for your users to load into their web
browsers in order to securely access your NiFi. This is done in much of the
same way as you created your server keys:
1. openssl genrsa -out <user1>.key 2048
2. openssl req -new -key <user1>.key -out <user1>.csr
3. openssl x509 -req -in <user1>.csr -CA rootCA.pem -CAkey rootCA.key
-CAcreateserial -out <user1>.crt -days 800
4. openssl pkcs12 -export -out <user1>.p12 -inkey <user1>.key -in
<user1>.crt -certfile rootCA.srl
Now you have a p12 file for user1. they can load this in to their browser
certs to use to authenticate against your secure NiFi.
Now remember you must manually add that first "ROLE_ADMIN" user to the
authorized-users.xml file. So you will need the DN from the user key you
created for this Admin user and add it in to your Authorized-users.xml
file.
Here is an example of what it may look like:
dn="EMAILADDRESS=none@none.com, CN=<user1>, OU=NiFi, O=someplace,
L=Baltimore, ST=Maryland, C=US"
If you have the DN format wrong in your authorized-users.xml file, rather
then gaining access to the NiFi you will get prompted to "request access”.
Do *Not* click the request access link. You must instead go fix the DN in
the authorized-users.xml file. You need to create that first admin account
that can approve those requests. If you click request access , you will
need to stop your NiFi and delete the user db, otherwise, even fixing your
authorized-usesr.xml file will not gain you access because your account
will be stuck in a pending auth state.
You can look at the request that came in in the nifi-users.log to get the
exact DN pattern to fix your authorized-usesr.xml file entry:
You should see something that looks like this:
INFO [NiFi Web Server-58023] o.a.n.w.s.x509.X509AuthenticationFilter
Attempting request for (<CN=John
Doe, OU=bobthebuilder, O=carpenter, L=Houston, S=Texas, C=US>) GET...
That log line gives you the exact format of the DN that needs to be
updated/added to the authorized-users.xml file. Example below:
<user dn="CN=John
Doe, OU=bobthebuilder, O=carpenter, L=Houston, S=Texas, C=US">
<role name="ROLE_DFM"/>
<role name="ROLE_ADMIN"/>
<role name="ROLE_PROVENANCE"/>
</user>
YOu only need to do the above if you are not using LDAP for user
authentication.
On Mon, Feb 15, 2016 at 2:55 PM, Joe Skora <js...@gmail.com> wrote:
> Matt,
>
> This is really helpful. If I'm not using LDAP, is a user key created the
> same way as a server/node key or is there another process for user keys?
>
> Thanks,
> Joe
>
> On Mon, Feb 15, 2016 at 12:12 PM, Matthew Clarke <
> matt.clarke.138@gmail.com> wrote:
>
>> The communications between secure NiFi cluster Nodes uses 2-way SSL which
>> requires both keystores and truststores. The truststore typically includes
>> a bunch of Certificate Authorities, but in your case since you are using
>> self-signed certificates for each of your nodes, it will also need to
>> include the keys for everyone of your nodes. You can simply this by
>> creating your own CA to sign all certificates. Then you only need to add
>> your CA to each nodes truststore.
>>
>> You can use a site tinycert.org top create yoru won CA and certificates
>> for free or the following procedure may be helpful if you want to create
>> them locally yourself:
>>
>>
>> You only need to create one Certificate Authority (CA), which you will
>> use to sign the keys for every one of your servers/VMs and users (You only
>> need to create keys for
>> users if your NiFi has not been configured to use LDAP authentication).
>> What is a CA? The CA acts as a trusted entity for validating the
>> authenticity of certificates.
>>
>>
>>
>> Commands for creating a CA:
>>
>>
>>
>> 1. *openssl genrsa -out rootCA.key 2048 *
>>
>> 2. *openssl req -x509 -new -nodes -key rootCA.key -days 1024 -out
>> rootCA.pem *
>>
>> 3. *openssl x509 -outform der -in rootCA.pem -out rootCA.der *
>>
>> 4. *keytool -import -keystore cacerts.jks -file rootCA.der *
>>
>>
>>
>> At the end of the above you will have your "truststore" file
>> (cacerts.jks) that you will use in your nifi.properties file. Use this same
>> "truststore" file on every one of your servers/VMs.
>>
>>
>>
>> Now lets create a server/vm key and get it signed by that CA:
>>
>> These are the steps I follow:
>>
>>
>>
>> 1. *keytool -genkey -alias nifi-server1 -keyalg RSA -keystore
>> nifi-server1.jks -keysize 2048 *
>>
>> 2. *keytool -certreq -alias nifi-server1 -keystore nifi-server1.jks
>> -file nifi-server1.csr *
>>
>> 3. *openssl x509 -req -in nifi-server1.csr -CA rootCA.pem -CAkey
>> rootCA.key -CAcreateserial -out nifi-server1.crt -days 1000 *
>>
>> 4. *keytool -import -keystore nifi-server1.jks -file rootCA.pem *
>>
>> 5. *keytool -import -trustcacerts -alias nifi-server1 -file
>> nifi-server1.crt -keystore nifi-server1.jks *
>>
>>
>>
>> At the end of the above you will have your "keystore" file
>> (nifi-server1.jks) that you will use in your nifi.properties file for one
>> of your servers/VMs. You will need to repeat the above
>> steps for each of your other servers/VMs so they each use their own keys.
>> Now keep in mind that I am using “nifi-server1" in this example, but you
>> will most likely use your
>> systems/VMs hostnames (shortname as alias and FQDN as CN). I also highly
>> recommend that you use the same key and keystore password for every key you
>> create if creating
>> keys for multiple nodes in a NiFi cluster.
>>
>>
>>
>> On Mon, Feb 15, 2016 at 11:39 AM, Conrad Crampton <
>> conrad.crampton@secdata.com> wrote:
>>
>>> Hi,
>>> New to this list and loving NiFi so far ;-)
>>> I have set up and been trying out with good results a NiFi set up
>>> locally on my laptop – all good. Managed to set up SSL configuration and
>>> hook up with company LDAP.
>>>
>>> I have also set up three node cluster with cluster manager on a separate
>>> host, all using http – all good here too. NCM communicates fine with three
>>> nodes, and vice versa. Ok, moving on to set up cluster to use SSL and LDAP
>>> is where I have come a bit unstuck.
>>>
>>> I am using self signed certificate – on laptop used one generated using
>>> openssl, then converted to pks file (which worked fine as above).
>>>
>>> The issue I’m having is running virtually the same setup on cluster as
>>> laptop fails with the following exception (there is loads of other nested
>>> exception details) but it appears to boil down to this one.
>>>
>>>
>>> Caused by: org.springframework.beans.factory.BeanCreationException:
>>> Error creating bean with name 'protocolSocketConfiguration': FactoryBean
>>> threw exception on object creation; nested exception is
>>> java.security.KeyStoreException: not found
>>> at
>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175)
>>> ~[na:na]
>>> at
>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
>>> ~[na:na]
>>> at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1517)
>>> ~[na:na]
>>> at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:314)
>>> ~[na:na]
>>> at
>>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
>>> ~[na:na]
>>> at
>>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
>>> ~[na:na]
>>> ... 103 common frames omitted
>>> Caused by: java.security.KeyStoreException: not found
>>> at java.security.KeyStore.getInstance(KeyStore.java:851)
>>> ~[na:1.8.0_51]
>>> at
>>> org.apache.nifi.io.socket.SSLContextFactory.<init>(SSLContextFactory.java:72)
>>> ~[na:na]
>>> at
>>> org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:46)
>>> ~[na:na]
>>> at
>>> org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:31)
>>> ~[na:na]
>>> at
>>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168)
>>> ~[na:na]
>>> ... 108 common frames omitted
>>> Caused by: java.security.NoSuchAlgorithmException: KeyStore not
>>> available
>>> at
>>> sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
>>> ~[na:1.8.0_51]
>>> at java.security.Security.getImpl(Security.java:695)
>>> ~[na:1.8.0_51]
>>> at java.security.KeyStore.getInstance(KeyStore.java:848)
>>> ~[na:1.8.0_51]
>>>
>>>
>>>
>>> Having looked at the code, line 72 (&73) of SSLContextFactory is
>>>
>>> final KeyStore trustStore = KeyStore.getInstance(truststoreType);
>>> final FileInputStream trustStoreStream = new FileInputStream(truststore);
>>>
>>> Which is throwing the error by the looks of it, so appears to need a
>>> value in the nifi.properties file for nifi.security.truststoreType= which
>>> on my laptop config is blank (docs seems to suggest that this is optional
>>> anyway), but the code complains about this not being present. If I enter a
>>> value (JKS or PKCS12) in here I get a filenotfound exception on line 73
>>> instead as I haven’t provided a value for truststore location. So, do I
>>> need to provide truststore configuration in a clustered environment? If so,
>>> docs would helpful to point this out, if not anyone suggest something to
>>> fix this please? It took me 10 mins to set up on laptop, now after 6 hours
>>> of trying on cluster I’m stumped.
>>>
>>> Thanks
>>> Conrad
>>>
>>>
>>> SecureData, combating cyber threats
>>>
>>> ------------------------------
>>>
>>> The information contained in this message or any of its attachments may
>>> be privileged and confidential and intended for the exclusive use of the
>>> intended recipient. If you are not the intended recipient any disclosure,
>>> reproduction, distribution or other dissemination or use of this
>>> communications is strictly prohibited. The views expressed in this email
>>> are those of the individual and not necessarily of SecureData Europe Ltd.
>>> Any prices quoted are only valid if followed up by a formal written quote.
>>>
>>> SecureData Europe Limited. Registered in England & Wales 04365896.
>>> Registered Address: SecureData House, Hermitage Court, Hermitage Lane,
>>> Maidstone, Kent, ME16 9NT
>>>
>>
>>
>
Re: SSL configuration on cluster
Posted by Joe Skora <js...@gmail.com>.
Matt,
This is really helpful. If I'm not using LDAP, is a user key created the
same way as a server/node key or is there another process for user keys?
Thanks,
Joe
On Mon, Feb 15, 2016 at 12:12 PM, Matthew Clarke <ma...@gmail.com>
wrote:
> The communications between secure NiFi cluster Nodes uses 2-way SSL which
> requires both keystores and truststores. The truststore typically includes
> a bunch of Certificate Authorities, but in your case since you are using
> self-signed certificates for each of your nodes, it will also need to
> include the keys for everyone of your nodes. You can simply this by
> creating your own CA to sign all certificates. Then you only need to add
> your CA to each nodes truststore.
>
> You can use a site tinycert.org top create yoru won CA and certificates
> for free or the following procedure may be helpful if you want to create
> them locally yourself:
>
>
> You only need to create one Certificate Authority (CA), which you will use
> to sign the keys for every one of your servers/VMs and users (You only need
> to create keys for
> users if your NiFi has not been configured to use LDAP authentication).
> What is a CA? The CA acts as a trusted entity for validating the
> authenticity of certificates.
>
>
>
> Commands for creating a CA:
>
>
>
> 1. *openssl genrsa -out rootCA.key 2048 *
>
> 2. *openssl req -x509 -new -nodes -key rootCA.key -days 1024 -out
> rootCA.pem *
>
> 3. *openssl x509 -outform der -in rootCA.pem -out rootCA.der *
>
> 4. *keytool -import -keystore cacerts.jks -file rootCA.der *
>
>
>
> At the end of the above you will have your "truststore" file (cacerts.jks)
> that you will use in your nifi.properties file. Use this same "truststore"
> file on every one of your servers/VMs.
>
>
>
> Now lets create a server/vm key and get it signed by that CA:
>
> These are the steps I follow:
>
>
>
> 1. *keytool -genkey -alias nifi-server1 -keyalg RSA -keystore
> nifi-server1.jks -keysize 2048 *
>
> 2. *keytool -certreq -alias nifi-server1 -keystore nifi-server1.jks -file
> nifi-server1.csr *
>
> 3. *openssl x509 -req -in nifi-server1.csr -CA rootCA.pem -CAkey
> rootCA.key -CAcreateserial -out nifi-server1.crt -days 1000 *
>
> 4. *keytool -import -keystore nifi-server1.jks -file rootCA.pem *
>
> 5. *keytool -import -trustcacerts -alias nifi-server1 -file
> nifi-server1.crt -keystore nifi-server1.jks *
>
>
>
> At the end of the above you will have your "keystore" file
> (nifi-server1.jks) that you will use in your nifi.properties file for one
> of your servers/VMs. You will need to repeat the above
> steps for each of your other servers/VMs so they each use their own keys.
> Now keep in mind that I am using “nifi-server1" in this example, but you
> will most likely use your
> systems/VMs hostnames (shortname as alias and FQDN as CN). I also highly
> recommend that you use the same key and keystore password for every key you
> create if creating
> keys for multiple nodes in a NiFi cluster.
>
>
>
> On Mon, Feb 15, 2016 at 11:39 AM, Conrad Crampton <
> conrad.crampton@secdata.com> wrote:
>
>> Hi,
>> New to this list and loving NiFi so far ;-)
>> I have set up and been trying out with good results a NiFi set up locally
>> on my laptop – all good. Managed to set up SSL configuration and hook up
>> with company LDAP.
>>
>> I have also set up three node cluster with cluster manager on a separate
>> host, all using http – all good here too. NCM communicates fine with three
>> nodes, and vice versa. Ok, moving on to set up cluster to use SSL and LDAP
>> is where I have come a bit unstuck.
>>
>> I am using self signed certificate – on laptop used one generated using
>> openssl, then converted to pks file (which worked fine as above).
>>
>> The issue I’m having is running virtually the same setup on cluster as
>> laptop fails with the following exception (there is loads of other nested
>> exception details) but it appears to boil down to this one.
>>
>>
>> Caused by: org.springframework.beans.factory.BeanCreationException: Error
>> creating bean with name 'protocolSocketConfiguration': FactoryBean threw
>> exception on object creation; nested exception is
>> java.security.KeyStoreException: not found
>> at
>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175)
>> ~[na:na]
>> at
>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
>> ~[na:na]
>> at
>> org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1517)
>> ~[na:na]
>> at
>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:314)
>> ~[na:na]
>> at
>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
>> ~[na:na]
>> at
>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
>> ~[na:na]
>> ... 103 common frames omitted
>> Caused by: java.security.KeyStoreException: not found
>> at java.security.KeyStore.getInstance(KeyStore.java:851)
>> ~[na:1.8.0_51]
>> at
>> org.apache.nifi.io.socket.SSLContextFactory.<init>(SSLContextFactory.java:72)
>> ~[na:na]
>> at
>> org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:46)
>> ~[na:na]
>> at
>> org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:31)
>> ~[na:na]
>> at
>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168)
>> ~[na:na]
>> ... 108 common frames omitted
>> Caused by: java.security.NoSuchAlgorithmException: KeyStore not available
>> at sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
>> ~[na:1.8.0_51]
>> at java.security.Security.getImpl(Security.java:695)
>> ~[na:1.8.0_51]
>> at java.security.KeyStore.getInstance(KeyStore.java:848)
>> ~[na:1.8.0_51]
>>
>>
>>
>> Having looked at the code, line 72 (&73) of SSLContextFactory is
>>
>> final KeyStore trustStore = KeyStore.getInstance(truststoreType);
>> final FileInputStream trustStoreStream = new FileInputStream(truststore);
>>
>> Which is throwing the error by the looks of it, so appears to need a
>> value in the nifi.properties file for nifi.security.truststoreType= which
>> on my laptop config is blank (docs seems to suggest that this is optional
>> anyway), but the code complains about this not being present. If I enter a
>> value (JKS or PKCS12) in here I get a filenotfound exception on line 73
>> instead as I haven’t provided a value for truststore location. So, do I
>> need to provide truststore configuration in a clustered environment? If so,
>> docs would helpful to point this out, if not anyone suggest something to
>> fix this please? It took me 10 mins to set up on laptop, now after 6 hours
>> of trying on cluster I’m stumped.
>>
>> Thanks
>> Conrad
>>
>>
>> SecureData, combating cyber threats
>>
>> ------------------------------
>>
>> The information contained in this message or any of its attachments may
>> be privileged and confidential and intended for the exclusive use of the
>> intended recipient. If you are not the intended recipient any disclosure,
>> reproduction, distribution or other dissemination or use of this
>> communications is strictly prohibited. The views expressed in this email
>> are those of the individual and not necessarily of SecureData Europe Ltd.
>> Any prices quoted are only valid if followed up by a formal written quote.
>>
>> SecureData Europe Limited. Registered in England & Wales 04365896.
>> Registered Address: SecureData House, Hermitage Court, Hermitage Lane,
>> Maidstone, Kent, ME16 9NT
>>
>
>
Re: SSL configuration on cluster
Posted by Conrad Crampton <co...@SecData.com>.
Matt,
Incredibly helpful thanks. I’ll give this a go.
Probably something that should go in the docs too as without out this, very difficult to figure out - for me anyway ;-)
Thanks
Conrad
From: Matthew Clarke <ma...@gmail.com>>
Reply-To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Date: Monday, 15 February 2016 at 17:12
To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Subject: Re: SSL configuration on cluster
The communications between secure NiFi cluster Nodes uses 2-way SSL which requires both keystores and truststores. The truststore typically includes a bunch of Certificate Authorities, but in your case since you are using self-signed certificates for each of your nodes, it will also need to include the keys for everyone of your nodes. You can simply this by creating your own CA to sign all certificates. Then you only need to add your CA to each nodes truststore.
You can use a site tinycert.org<http://tinycert.org> top create yoru won CA and certificates for free or the following procedure may be helpful if you want to create them locally yourself:
You only need to create one Certificate Authority (CA), which you will use to sign the keys for every one of your servers/VMs and users (You only need to create keys for
users if your NiFi has not been configured to use LDAP authentication). What is a CA? The CA acts as a trusted entity for validating the authenticity of certificates.
Commands for creating a CA:
1. openssl genrsa -out rootCA.key 2048
2. openssl req -x509 -new -nodes -key rootCA.key -days 1024 -out rootCA.pem
3. openssl x509 -outform der -in rootCA.pem -out rootCA.der
4. keytool -import -keystore cacerts.jks -file rootCA.der
At the end of the above you will have your "truststore" file (cacerts.jks) that you will use in your nifi.properties file. Use this same "truststore" file on every one of your servers/VMs.
Now lets create a server/vm key and get it signed by that CA:
These are the steps I follow:
1. keytool -genkey -alias nifi-server1 -keyalg RSA -keystore nifi-server1.jks -keysize 2048
2. keytool -certreq -alias nifi-server1 -keystore nifi-server1.jks -file nifi-server1.csr
3. openssl x509 -req -in nifi-server1.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out nifi-server1.crt -days 1000
4. keytool -import -keystore nifi-server1.jks -file rootCA.pem
5. keytool -import -trustcacerts -alias nifi-server1 -file nifi-server1.crt -keystore nifi-server1.jks
At the end of the above you will have your "keystore" file (nifi-server1.jks) that you will use in your nifi.properties file for one of your servers/VMs. You will need to repeat the above
steps for each of your other servers/VMs so they each use their own keys. Now keep in mind that I am using “nifi-server1" in this example, but you will most likely use your
systems/VMs hostnames (shortname as alias and FQDN as CN). I also highly recommend that you use the same key and keystore password for every key you create if creating
keys for multiple nodes in a NiFi cluster.
On Mon, Feb 15, 2016 at 11:39 AM, Conrad Crampton <co...@secdata.com>> wrote:
Hi,
New to this list and loving NiFi so far ;-)
I have set up and been trying out with good results a NiFi set up locally on my laptop – all good. Managed to set up SSL configuration and hook up with company LDAP.
I have also set up three node cluster with cluster manager on a separate host, all using http – all good here too. NCM communicates fine with three nodes, and vice versa. Ok, moving on to set up cluster to use SSL and LDAP is where I have come a bit unstuck.
I am using self signed certificate – on laptop used one generated using openssl, then converted to pks file (which worked fine as above).
The issue I’m having is running virtually the same setup on cluster as laptop fails with the following exception (there is loads of other nested exception details) but it appears to boil down to this one.
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'protocolSocketConfiguration': FactoryBean threw exception on object creation; nested exception is java.security.KeyStoreException: not found
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175) ~[na:na]
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) ~[na:na]
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1517) ~[na:na]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:314) ~[na:na]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194) ~[na:na]
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351) ~[na:na]
... 103 common frames omitted
Caused by: java.security.KeyStoreException: not found
at java.security.KeyStore.getInstance(KeyStore.java:851) ~[na:1.8.0_51]
at org.apache.nifi.io.socket.SSLContextFactory.<init>(SSLContextFactory.java:72) ~[na:na]
at org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:46) ~[na:na]
at org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:31) ~[na:na]
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) ~[na:na]
... 108 common frames omitted
Caused by: java.security.NoSuchAlgorithmException: KeyStore not available
at sun.security.jca.GetInstance.getInstance(GetInstance.java:159) ~[na:1.8.0_51]
at java.security.Security.getImpl(Security.java:695) ~[na:1.8.0_51]
at java.security.KeyStore.getInstance(KeyStore.java:848) ~[na:1.8.0_51]
Having looked at the code, line 72 (&73) of SSLContextFactory is
final KeyStore trustStore = KeyStore.getInstance(truststoreType);
final FileInputStream trustStoreStream = new FileInputStream(truststore);
Which is throwing the error by the looks of it, so appears to need a value in the nifi.properties file for nifi.security.truststoreType= which on my laptop config is blank (docs seems to suggest that this is optional anyway), but the code complains about this not being present. If I enter a value (JKS or PKCS12) in here I get a filenotfound exception on line 73 instead as I haven’t provided a value for truststore location. So, do I need to provide truststore configuration in a clustered environment? If so, docs would helpful to point this out, if not anyone suggest something to fix this please? It took me 10 mins to set up on laptop, now after 6 hours of trying on cluster I’m stumped.
Thanks
Conrad
SecureData, combating cyber threats
________________________________
The information contained in this message or any of its attachments may be privileged and confidential and intended for the exclusive use of the intended recipient. If you are not the intended recipient any disclosure, reproduction, distribution or other dissemination or use of this communications is strictly prohibited. The views expressed in this email are those of the individual and not necessarily of SecureData Europe Ltd. Any prices quoted are only valid if followed up by a formal written quote.
SecureData Europe Limited. Registered in England & Wales 04365896. Registered Address: SecureData House, Hermitage Court, Hermitage Lane, Maidstone, Kent, ME16 9NT
***This email originated outside SecureData***
Click here<https://www.mailcontrol.com/sr/Ni4e5Ve2jFPGX2PQPOmvUhkLFoJbzkFEUGpOKun7w!QfrdtWVdOniSZyTuGcCNTxpHEUYPvxwk+TKOxjbsImtQ==> to report this email as spam.
Re: SSL configuration on cluster
Posted by Matthew Clarke <ma...@gmail.com>.
Thank you
On Tue, Feb 16, 2016 at 7:56 AM, Conrad Crampton <
conrad.crampton@secdata.com> wrote:
> Hi All,
> For the record – these instructions are spot on and worked like a charm –
> thanks Matt.
> Conrad
>
> From: Matthew Clarke <ma...@gmail.com>
> Reply-To: "users@nifi.apache.org" <us...@nifi.apache.org>
> Date: Monday, 15 February 2016 at 17:12
> To: "users@nifi.apache.org" <us...@nifi.apache.org>
> Subject: Re: SSL configuration on cluster
>
> The communications between secure NiFi cluster Nodes uses 2-way SSL which
> requires both keystores and truststores. The truststore typically includes
> a bunch of Certificate Authorities, but in your case since you are using
> self-signed certificates for each of your nodes, it will also need to
> include the keys for everyone of your nodes. You can simply this by
> creating your own CA to sign all certificates. Then you only need to add
> your CA to each nodes truststore.
>
> You can use a site tinycert.org top create yoru won CA and certificates
> for free or the following procedure may be helpful if you want to create
> them locally yourself:
>
>
> You only need to create one Certificate Authority (CA), which you will use
> to sign the keys for every one of your servers/VMs and users (You only need
> to create keys for
> users if your NiFi has not been configured to use LDAP authentication).
> What is a CA? The CA acts as a trusted entity for validating the
> authenticity of certificates.
>
>
>
> Commands for creating a CA:
>
>
>
> 1. *openssl genrsa -out rootCA.key 2048 *
>
> 2. *openssl req -x509 -new -nodes -key rootCA.key -days 1024 -out
> rootCA.pem *
>
> 3. *openssl x509 -outform der -in rootCA.pem -out rootCA.der *
>
> 4. *keytool -import -keystore cacerts.jks -file rootCA.der *
>
>
>
> At the end of the above you will have your "truststore" file (cacerts.jks)
> that you will use in your nifi.properties file. Use this same "truststore"
> file on every one of your servers/VMs.
>
>
>
> Now lets create a server/vm key and get it signed by that CA:
>
> These are the steps I follow:
>
>
>
> 1. *keytool -genkey -alias nifi-server1 -keyalg RSA -keystore
> nifi-server1.jks -keysize 2048 *
>
> 2. *keytool -certreq -alias nifi-server1 -keystore nifi-server1.jks -file
> nifi-server1.csr *
>
> 3. *openssl x509 -req -in nifi-server1.csr -CA rootCA.pem -CAkey
> rootCA.key -CAcreateserial -out nifi-server1.crt -days 1000 *
>
> 4. *keytool -import -keystore nifi-server1.jks -file rootCA.pem *
>
> 5. *keytool -import -trustcacerts -alias nifi-server1 -file
> nifi-server1.crt -keystore nifi-server1.jks *
>
>
>
> At the end of the above you will have your "keystore" file
> (nifi-server1.jks) that you will use in your nifi.properties file for one
> of your servers/VMs. You will need to repeat the above
> steps for each of your other servers/VMs so they each use their own keys.
> Now keep in mind that I am using “nifi-server1" in this example, but you
> will most likely use your
> systems/VMs hostnames (shortname as alias and FQDN as CN). I also highly
> recommend that you use the same key and keystore password for every key you
> create if creating
> keys for multiple nodes in a NiFi cluster.
>
>
>
> On Mon, Feb 15, 2016 at 11:39 AM, Conrad Crampton <
> conrad.crampton@secdata.com> wrote:
>
>> Hi,
>> New to this list and loving NiFi so far ;-)
>> I have set up and been trying out with good results a NiFi set up locally
>> on my laptop – all good. Managed to set up SSL configuration and hook up
>> with company LDAP.
>>
>> I have also set up three node cluster with cluster manager on a separate
>> host, all using http – all good here too. NCM communicates fine with three
>> nodes, and vice versa. Ok, moving on to set up cluster to use SSL and LDAP
>> is where I have come a bit unstuck.
>>
>> I am using self signed certificate – on laptop used one generated using
>> openssl, then converted to pks file (which worked fine as above).
>>
>> The issue I’m having is running virtually the same setup on cluster as
>> laptop fails with the following exception (there is loads of other nested
>> exception details) but it appears to boil down to this one.
>>
>>
>> Caused by: org.springframework.beans.factory.BeanCreationException: Error
>> creating bean with name 'protocolSocketConfiguration': FactoryBean threw
>> exception on object creation; nested exception is
>> java.security.KeyStoreException: not found
>> at
>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175)
>> ~[na:na]
>> at
>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
>> ~[na:na]
>> at
>> org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1517)
>> ~[na:na]
>> at
>> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:314)
>> ~[na:na]
>> at
>> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
>> ~[na:na]
>> at
>> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
>> ~[na:na]
>> ... 103 common frames omitted
>> Caused by: java.security.KeyStoreException: not found
>> at java.security.KeyStore.getInstance(KeyStore.java:851)
>> ~[na:1.8.0_51]
>> at
>> org.apache.nifi.io.socket.SSLContextFactory.<init>(SSLContextFactory.java:72)
>> ~[na:na]
>> at
>> org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:46)
>> ~[na:na]
>> at
>> org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:31)
>> ~[na:na]
>> at
>> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168)
>> ~[na:na]
>> ... 108 common frames omitted
>> Caused by: java.security.NoSuchAlgorithmException: KeyStore not available
>> at sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
>> ~[na:1.8.0_51]
>> at java.security.Security.getImpl(Security.java:695)
>> ~[na:1.8.0_51]
>> at java.security.KeyStore.getInstance(KeyStore.java:848)
>> ~[na:1.8.0_51]
>>
>>
>>
>> Having looked at the code, line 72 (&73) of SSLContextFactory is
>>
>> final KeyStore trustStore = KeyStore.getInstance(truststoreType);
>> final FileInputStream trustStoreStream = new FileInputStream(truststore);
>>
>> Which is throwing the error by the looks of it, so appears to need a
>> value in the nifi.properties file for nifi.security.truststoreType= which
>> on my laptop config is blank (docs seems to suggest that this is optional
>> anyway), but the code complains about this not being present. If I enter a
>> value (JKS or PKCS12) in here I get a filenotfound exception on line 73
>> instead as I haven’t provided a value for truststore location. So, do I
>> need to provide truststore configuration in a clustered environment? If so,
>> docs would helpful to point this out, if not anyone suggest something to
>> fix this please? It took me 10 mins to set up on laptop, now after 6 hours
>> of trying on cluster I’m stumped.
>>
>> Thanks
>> Conrad
>>
>>
>> SecureData, combating cyber threats
>>
>> ------------------------------
>>
>> The information contained in this message or any of its attachments may
>> be privileged and confidential and intended for the exclusive use of the
>> intended recipient. If you are not the intended recipient any disclosure,
>> reproduction, distribution or other dissemination or use of this
>> communications is strictly prohibited. The views expressed in this email
>> are those of the individual and not necessarily of SecureData Europe Ltd.
>> Any prices quoted are only valid if followed up by a formal written quote.
>>
>> SecureData Europe Limited. Registered in England & Wales 04365896.
>> Registered Address: SecureData House, Hermitage Court, Hermitage Lane,
>> Maidstone, Kent, ME16 9NT
>>
>
>
>
> ***This email originated outside SecureData***
>
> Click here
> <https://www.mailcontrol.com/sr/Ni4e5Ve2jFPGX2PQPOmvUhkLFoJbzkFEUGpOKun7w!QfrdtWVdOniSZyTuGcCNTxpHEUYPvxwk+TKOxjbsImtQ==>
> to report this email as spam.
>
Re: SSL configuration on cluster
Posted by Conrad Crampton <co...@SecData.com>.
Hi All,
For the record – these instructions are spot on and worked like a charm – thanks Matt.
Conrad
From: Matthew Clarke <ma...@gmail.com>>
Reply-To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Date: Monday, 15 February 2016 at 17:12
To: "users@nifi.apache.org<ma...@nifi.apache.org>" <us...@nifi.apache.org>>
Subject: Re: SSL configuration on cluster
The communications between secure NiFi cluster Nodes uses 2-way SSL which requires both keystores and truststores. The truststore typically includes a bunch of Certificate Authorities, but in your case since you are using self-signed certificates for each of your nodes, it will also need to include the keys for everyone of your nodes. You can simply this by creating your own CA to sign all certificates. Then you only need to add your CA to each nodes truststore.
You can use a site tinycert.org<http://tinycert.org> top create yoru won CA and certificates for free or the following procedure may be helpful if you want to create them locally yourself:
You only need to create one Certificate Authority (CA), which you will use to sign the keys for every one of your servers/VMs and users (You only need to create keys for
users if your NiFi has not been configured to use LDAP authentication). What is a CA? The CA acts as a trusted entity for validating the authenticity of certificates.
Commands for creating a CA:
1. openssl genrsa -out rootCA.key 2048
2. openssl req -x509 -new -nodes -key rootCA.key -days 1024 -out rootCA.pem
3. openssl x509 -outform der -in rootCA.pem -out rootCA.der
4. keytool -import -keystore cacerts.jks -file rootCA.der
At the end of the above you will have your "truststore" file (cacerts.jks) that you will use in your nifi.properties file. Use this same "truststore" file on every one of your servers/VMs.
Now lets create a server/vm key and get it signed by that CA:
These are the steps I follow:
1. keytool -genkey -alias nifi-server1 -keyalg RSA -keystore nifi-server1.jks -keysize 2048
2. keytool -certreq -alias nifi-server1 -keystore nifi-server1.jks -file nifi-server1.csr
3. openssl x509 -req -in nifi-server1.csr -CA rootCA.pem -CAkey rootCA.key -CAcreateserial -out nifi-server1.crt -days 1000
4. keytool -import -keystore nifi-server1.jks -file rootCA.pem
5. keytool -import -trustcacerts -alias nifi-server1 -file nifi-server1.crt -keystore nifi-server1.jks
At the end of the above you will have your "keystore" file (nifi-server1.jks) that you will use in your nifi.properties file for one of your servers/VMs. You will need to repeat the above
steps for each of your other servers/VMs so they each use their own keys. Now keep in mind that I am using “nifi-server1" in this example, but you will most likely use your
systems/VMs hostnames (shortname as alias and FQDN as CN). I also highly recommend that you use the same key and keystore password for every key you create if creating
keys for multiple nodes in a NiFi cluster.
On Mon, Feb 15, 2016 at 11:39 AM, Conrad Crampton <co...@secdata.com>> wrote:
Hi,
New to this list and loving NiFi so far ;-)
I have set up and been trying out with good results a NiFi set up locally on my laptop – all good. Managed to set up SSL configuration and hook up with company LDAP.
I have also set up three node cluster with cluster manager on a separate host, all using http – all good here too. NCM communicates fine with three nodes, and vice versa. Ok, moving on to set up cluster to use SSL and LDAP is where I have come a bit unstuck.
I am using self signed certificate – on laptop used one generated using openssl, then converted to pks file (which worked fine as above).
The issue I’m having is running virtually the same setup on cluster as laptop fails with the following exception (there is loads of other nested exception details) but it appears to boil down to this one.
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'protocolSocketConfiguration': FactoryBean threw exception on object creation; nested exception is java.security.KeyStoreException: not found
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175) ~[na:na]
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103) ~[na:na]
at org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1517) ~[na:na]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:314) ~[na:na]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194) ~[na:na]
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351) ~[na:na]
... 103 common frames omitted
Caused by: java.security.KeyStoreException: not found
at java.security.KeyStore.getInstance(KeyStore.java:851) ~[na:1.8.0_51]
at org.apache.nifi.io.socket.SSLContextFactory.<init>(SSLContextFactory.java:72) ~[na:na]
at org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:46) ~[na:na]
at org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:31) ~[na:na]
at org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) ~[na:na]
... 108 common frames omitted
Caused by: java.security.NoSuchAlgorithmException: KeyStore not available
at sun.security.jca.GetInstance.getInstance(GetInstance.java:159) ~[na:1.8.0_51]
at java.security.Security.getImpl(Security.java:695) ~[na:1.8.0_51]
at java.security.KeyStore.getInstance(KeyStore.java:848) ~[na:1.8.0_51]
Having looked at the code, line 72 (&73) of SSLContextFactory is
final KeyStore trustStore = KeyStore.getInstance(truststoreType);
final FileInputStream trustStoreStream = new FileInputStream(truststore);
Which is throwing the error by the looks of it, so appears to need a value in the nifi.properties file for nifi.security.truststoreType= which on my laptop config is blank (docs seems to suggest that this is optional anyway), but the code complains about this not being present. If I enter a value (JKS or PKCS12) in here I get a filenotfound exception on line 73 instead as I haven’t provided a value for truststore location. So, do I need to provide truststore configuration in a clustered environment? If so, docs would helpful to point this out, if not anyone suggest something to fix this please? It took me 10 mins to set up on laptop, now after 6 hours of trying on cluster I’m stumped.
Thanks
Conrad
SecureData, combating cyber threats
________________________________
The information contained in this message or any of its attachments may be privileged and confidential and intended for the exclusive use of the intended recipient. If you are not the intended recipient any disclosure, reproduction, distribution or other dissemination or use of this communications is strictly prohibited. The views expressed in this email are those of the individual and not necessarily of SecureData Europe Ltd. Any prices quoted are only valid if followed up by a formal written quote.
SecureData Europe Limited. Registered in England & Wales 04365896. Registered Address: SecureData House, Hermitage Court, Hermitage Lane, Maidstone, Kent, ME16 9NT
***This email originated outside SecureData***
Click here<https://www.mailcontrol.com/sr/Ni4e5Ve2jFPGX2PQPOmvUhkLFoJbzkFEUGpOKun7w!QfrdtWVdOniSZyTuGcCNTxpHEUYPvxwk+TKOxjbsImtQ==> to report this email as spam.
Re: SSL configuration on cluster
Posted by Matthew Clarke <ma...@gmail.com>.
The communications between secure NiFi cluster Nodes uses 2-way SSL which
requires both keystores and truststores. The truststore typically includes
a bunch of Certificate Authorities, but in your case since you are using
self-signed certificates for each of your nodes, it will also need to
include the keys for everyone of your nodes. You can simply this by
creating your own CA to sign all certificates. Then you only need to add
your CA to each nodes truststore.
You can use a site tinycert.org top create yoru won CA and certificates for
free or the following procedure may be helpful if you want to create them
locally yourself:
You only need to create one Certificate Authority (CA), which you will use
to sign the keys for every one of your servers/VMs and users (You only need
to create keys for
users if your NiFi has not been configured to use LDAP authentication).
What is a CA? The CA acts as a trusted entity for validating the
authenticity of certificates.
Commands for creating a CA:
1. *openssl genrsa -out rootCA.key 2048 *
2. *openssl req -x509 -new -nodes -key rootCA.key -days 1024 -out
rootCA.pem *
3. *openssl x509 -outform der -in rootCA.pem -out rootCA.der *
4. *keytool -import -keystore cacerts.jks -file rootCA.der *
At the end of the above you will have your "truststore" file (cacerts.jks)
that you will use in your nifi.properties file. Use this same "truststore"
file on every one of your servers/VMs.
Now lets create a server/vm key and get it signed by that CA:
These are the steps I follow:
1. *keytool -genkey -alias nifi-server1 -keyalg RSA -keystore
nifi-server1.jks -keysize 2048 *
2. *keytool -certreq -alias nifi-server1 -keystore nifi-server1.jks -file
nifi-server1.csr *
3. *openssl x509 -req -in nifi-server1.csr -CA rootCA.pem -CAkey rootCA.key
-CAcreateserial -out nifi-server1.crt -days 1000 *
4. *keytool -import -keystore nifi-server1.jks -file rootCA.pem *
5. *keytool -import -trustcacerts -alias nifi-server1 -file
nifi-server1.crt -keystore nifi-server1.jks *
At the end of the above you will have your "keystore" file
(nifi-server1.jks) that you will use in your nifi.properties file for one
of your servers/VMs. You will need to repeat the above
steps for each of your other servers/VMs so they each use their own keys.
Now keep in mind that I am using “nifi-server1" in this example, but you
will most likely use your
systems/VMs hostnames (shortname as alias and FQDN as CN). I also highly
recommend that you use the same key and keystore password for every key you
create if creating
keys for multiple nodes in a NiFi cluster.
On Mon, Feb 15, 2016 at 11:39 AM, Conrad Crampton <
conrad.crampton@secdata.com> wrote:
> Hi,
> New to this list and loving NiFi so far ;-)
> I have set up and been trying out with good results a NiFi set up locally
> on my laptop – all good. Managed to set up SSL configuration and hook up
> with company LDAP.
>
> I have also set up three node cluster with cluster manager on a separate
> host, all using http – all good here too. NCM communicates fine with three
> nodes, and vice versa. Ok, moving on to set up cluster to use SSL and LDAP
> is where I have come a bit unstuck.
>
> I am using self signed certificate – on laptop used one generated using
> openssl, then converted to pks file (which worked fine as above).
>
> The issue I’m having is running virtually the same setup on cluster as
> laptop fails with the following exception (there is loads of other nested
> exception details) but it appears to boil down to this one.
>
>
> Caused by: org.springframework.beans.factory.BeanCreationException: Error
> creating bean with name 'protocolSocketConfiguration': FactoryBean threw
> exception on object creation; nested exception is
> java.security.KeyStoreException: not found
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:175)
> ~[na:na]
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.getObjectFromFactoryBean(FactoryBeanRegistrySupport.java:103)
> ~[na:na]
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getObjectForBeanInstance(AbstractBeanFactory.java:1517)
> ~[na:na]
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:314)
> ~[na:na]
> at
> org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
> ~[na:na]
> at
> org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)
> ~[na:na]
> ... 103 common frames omitted
> Caused by: java.security.KeyStoreException: not found
> at java.security.KeyStore.getInstance(KeyStore.java:851)
> ~[na:1.8.0_51]
> at
> org.apache.nifi.io.socket.SSLContextFactory.<init>(SSLContextFactory.java:72)
> ~[na:na]
> at
> org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:46)
> ~[na:na]
> at
> org.apache.nifi.cluster.protocol.spring.SocketConfigurationFactoryBean.getObject(SocketConfigurationFactoryBean.java:31)
> ~[na:na]
> at
> org.springframework.beans.factory.support.FactoryBeanRegistrySupport.doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168)
> ~[na:na]
> ... 108 common frames omitted
> Caused by: java.security.NoSuchAlgorithmException: KeyStore not available
> at sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
> ~[na:1.8.0_51]
> at java.security.Security.getImpl(Security.java:695) ~[na:1.8.0_51]
> at java.security.KeyStore.getInstance(KeyStore.java:848)
> ~[na:1.8.0_51]
>
>
>
> Having looked at the code, line 72 (&73) of SSLContextFactory is
>
> final KeyStore trustStore = KeyStore.getInstance(truststoreType);
> final FileInputStream trustStoreStream = new FileInputStream(truststore);
>
> Which is throwing the error by the looks of it, so appears to need a value
> in the nifi.properties file for nifi.security.truststoreType= which on my
> laptop config is blank (docs seems to suggest that this is optional
> anyway), but the code complains about this not being present. If I enter a
> value (JKS or PKCS12) in here I get a filenotfound exception on line 73
> instead as I haven’t provided a value for truststore location. So, do I
> need to provide truststore configuration in a clustered environment? If so,
> docs would helpful to point this out, if not anyone suggest something to
> fix this please? It took me 10 mins to set up on laptop, now after 6 hours
> of trying on cluster I’m stumped.
>
> Thanks
> Conrad
>
>
> SecureData, combating cyber threats
>
> ------------------------------
>
> The information contained in this message or any of its attachments may be
> privileged and confidential and intended for the exclusive use of the
> intended recipient. If you are not the intended recipient any disclosure,
> reproduction, distribution or other dissemination or use of this
> communications is strictly prohibited. The views expressed in this email
> are those of the individual and not necessarily of SecureData Europe Ltd.
> Any prices quoted are only valid if followed up by a formal written quote.
>
> SecureData Europe Limited. Registered in England & Wales 04365896.
> Registered Address: SecureData House, Hermitage Court, Hermitage Lane,
> Maidstone, Kent, ME16 9NT
>