You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Ken Danniswara <ke...@gmail.com> on 2019/07/01 06:45:09 UTC
Comments about installing Secured NiFi
Hello,
Couple days ago I talked with Andy(@yolopey) over twitter about me
experiencing a hard part when installing secure NiFi 1.9.2. Before I forgot
(as usual) I thought if I put it somewhere else it could be somehow
helpful.
First, not getting used with LDAP DN creates a long confusion. I have hard
time following example which DN tree to use on different parts of the
guide. While the LDAP tutorial is outside the scope, maybe having
consistent DN tree throughout the guide could be helpful. For example
between File-based (LDAP Authentication)
<https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#file-based-ldap-authentication>
and LDAP-based Users/Groups Referencing User DN
<https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#ldap-based-users-groups-referencing-user-dn>,
also
when generating initial admin cert with TLS-Toolkit.
Other problem with DN it is spaced-sensitive. I created the person
certificate without space: tls-toolkit.sh -C
'cn=nifiadmin,ou=users,dc=exa,dc=local', then copied the same string to the
"initial admin identity" properties. Apparently the certificate
auto-generated the space and my 'not-spaced' version authorization became
failed in login time. In the end I tried with changing the initial admin +
deleting users.xml or simply change the name inside users.xml file directly
both works.
Last part which is my mistake. I did un-comment the legacy FileAuthorizer
class at the bottom of the authorizer.xml file. I thought it will be the
same procedure to do like enabling ldap-provider in the
local-identity-provider.xml. I am not sure how easy other fall to this
mistake.
These are my main challenges over building the secured NiFi. The problem
maybe would happen for person without LDAP experience like me. Otherwise
there are no big problem. I haven't tried the Kerberos one which I'd love
to try other time.
Best Regards,
Ken
Re: Comments about installing Secured NiFi
Posted by Andrew Lim <an...@gmail.com>.
Hi Ken,
I just wanted to echo Andy’s thoughts and thank you for taking the time to log and report the installation issues you encountered. The details you provided will help other NiFi users in similar circumstances and highlight specific areas where the documentation can be improved.
As others share, hoping this thread can lead to a significant positive impact on the NiFI UX.
-Drew
> On Jul 1, 2019, at 3:47 AM, Andy LoPresto <al...@gmail.com> wrote:
>
> Thanks for documenting this Ken. You’re right that this is challenging and not user-friendly, especially for first-time users.
>
> The point about DN spacing is especially well-taken. I’m working on new docs for this, and I’ll share them soon and hope your feedback will be helpful to make this process much easier for users. Thanks.
>
> If anyone has more info to add for difficult use cases or unexpected problems, please add it here.
>
> Andy LoPresto
> alopresto@apache.org
> alopresto.apache@gmail.com
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
>
> On Jun 30, 2019, at 23:45, Ken Danniswara <ke...@gmail.com> wrote:
>
>> Hello,
>>
>> Couple days ago I talked with Andy(@yolopey) over twitter about me experiencing a hard part when installing secure NiFi 1.9.2. Before I forgot (as usual) I thought if I put it somewhere else it could be somehow helpful.
>>
>> First, not getting used with LDAP DN creates a long confusion. I have hard time following example which DN tree to use on different parts of the guide. While the LDAP tutorial is outside the scope, maybe having consistent DN tree throughout the guide could be helpful. For example between File-based (LDAP Authentication) and LDAP-based Users/Groups Referencing User DN, also when generating initial admin cert with TLS-Toolkit.
>>
>> Other problem with DN it is spaced-sensitive. I created the person certificate without space: tls-toolkit.sh -C 'cn=nifiadmin,ou=users,dc=exa,dc=local', then copied the same string to the "initial admin identity" properties. Apparently the certificate auto-generated the space and my 'not-spaced' version authorization became failed in login time. In the end I tried with changing the initial admin + deleting users.xml or simply change the name inside users.xml file directly both works.
>>
>> Last part which is my mistake. I did un-comment the legacy FileAuthorizer class at the bottom of the authorizer.xml file. I thought it will be the same procedure to do like enabling ldap-provider in the local-identity-provider.xml. I am not sure how easy other fall to this mistake.
>>
>> These are my main challenges over building the secured NiFi. The problem maybe would happen for person without LDAP experience like me. Otherwise there are no big problem. I haven't tried the Kerberos one which I'd love to try other time.
>>
>> Best Regards,
>> Ken
Re: Comments about installing Secured NiFi
Posted by Andy LoPresto <al...@gmail.com>.
Thanks for documenting this Ken. You’re right that this is challenging and not user-friendly, especially for first-time users.
The point about DN spacing is especially well-taken. I’m working on new docs for this, and I’ll share them soon and hope your feedback will be helpful to make this process much easier for users. Thanks.
If anyone has more info to add for difficult use cases or unexpected problems, please add it here.
Andy LoPresto
alopresto@apache.org
alopresto.apache@gmail.com
PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69
> On Jun 30, 2019, at 23:45, Ken Danniswara <ke...@gmail.com> wrote:
>
> Hello,
>
> Couple days ago I talked with Andy(@yolopey) over twitter about me experiencing a hard part when installing secure NiFi 1.9.2. Before I forgot (as usual) I thought if I put it somewhere else it could be somehow helpful.
>
> First, not getting used with LDAP DN creates a long confusion. I have hard time following example which DN tree to use on different parts of the guide. While the LDAP tutorial is outside the scope, maybe having consistent DN tree throughout the guide could be helpful. For example between File-based (LDAP Authentication) and LDAP-based Users/Groups Referencing User DN, also when generating initial admin cert with TLS-Toolkit.
>
> Other problem with DN it is spaced-sensitive. I created the person certificate without space: tls-toolkit.sh -C 'cn=nifiadmin,ou=users,dc=exa,dc=local', then copied the same string to the "initial admin identity" properties. Apparently the certificate auto-generated the space and my 'not-spaced' version authorization became failed in login time. In the end I tried with changing the initial admin + deleting users.xml or simply change the name inside users.xml file directly both works.
>
> Last part which is my mistake. I did un-comment the legacy FileAuthorizer class at the bottom of the authorizer.xml file. I thought it will be the same procedure to do like enabling ldap-provider in the local-identity-provider.xml. I am not sure how easy other fall to this mistake.
>
> These are my main challenges over building the secured NiFi. The problem maybe would happen for person without LDAP experience like me. Otherwise there are no big problem. I haven't tried the Kerberos one which I'd love to try other time.
>
> Best Regards,
> Ken
Re: Comments about installing Secured NiFi
Posted by Erik Anderson <ea...@pobox.com>.
Ken,
At Bloomberg we wrote a lot of shell scripting around the configuration of NiFi .xml files that configures all of NiFi, including the LDAP pieces
I took a lot of inspiration from https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/sh/ <https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/sh/update_login_providers.sh#L34>
LDAP in particular is here
https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/sh/update_login_providers.sh#L34
I have documented all of our NiFi Docker container startup variables here. Take a look, this is ALL of NiFi configuration parameters all documented in 1 place.
https://github.com/dprophet/nifi/blob/master/nifi-docker/dockerhub/configuration_of_nifi.md
This work is very Docker engine, kubernetes, and docker swarm compatible. Works like a charm for any environment we run NiFi in (secure and unsecure developer systems)
In particular for LDAP you set these environment variables (or Kubernetes/DockerSwarm)
export NIFI_AUTH=ldap
export NIFI_INITIAL_ADMIN_IDENTITY='insert_ldap_user_name_here'
export NIFI_LDAP_AUTHENTICATION_STRATEGY=SIMPLE
export NIFI_LDAP_MANAGER_DN='CN=myqueru,OU=FOO,DC=topsecret,DC=mycompany,DC=com'",
export NIFI_LDAP_MANAGER_PASSWORD=topsecretpassword
export NIFI_LDAP_URL='ldap://somewhere.mycompany.com'
export NIFI_LDAP_USER_SEARCH_FILTER='accountname={0}'
export NIFI_LDAP_USER_SEARCH_BASE": "'OU=accounts,DC=topsecret,DC=mycompany,DC=com'",
export NIFI_LDAP_IDENTITY_STRATEGY="USE_USERNAME",
export NIFI_LDAP_GROUP_FILTER="LDAP_ROLE_YOU_WANT_TO_PULL_USER_LIST_FROM",
export NIFI_LDAP_GROUP_SEARCH_BASE = "OU=ROLE-Data,OU=Groups,OU=mycompany,DC=topsecret,DC=mycompany,DC=com",
docker run --name nifi --env-file <(env | grep NIFI_) blah blah blah
And the whole system comes alive with only 11 environment variables. Fully integrated with LDAP users and groups.
If this work was put back into the community you could run the apache nifi container on dockerhub directly and it would magically work.
I can give this work back but I am unsure who in the mailing list to coordinate with or what people need. Additionally I am 100% sure that the NiFi-fn the groups are working on will need to do something similar to what we did.
Erik Anderson
Bloomberg
On Mon, Jul 1, 2019, at 2:45 AM, Ken Danniswara wrote:
> Hello,
>
> Couple days ago I talked with Andy(@yolopey) over twitter about me experiencing a hard part when installing secure NiFi 1.9.2. Before I forgot (as usual) I thought if I put it somewhere else it could be somehow helpful.
>
> First, not getting used with LDAP DN creates a long confusion. I have hard time following example which DN tree to use on different parts of the guide. While the LDAP tutorial is outside the scope, maybe having consistent DN tree throughout the guide could be helpful. For example between File-based (LDAP Authentication) <https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#file-based-ldap-authentication> and LDAP-based Users/Groups Referencing User DN <https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#ldap-based-users-groups-referencing-user-dn>, also when generating initial admin cert with TLS-Toolkit.
>
> Other problem with DN it is spaced-sensitive. I created the person certificate without space: tls-toolkit.sh -C 'cn=nifiadmin,ou=users,dc=exa,dc=local', then copied the same string to the "initial admin identity" properties. Apparently the certificate auto-generated the space and my 'not-spaced' version authorization became failed in login time. In the end I tried with changing the initial admin + deleting users.xml or simply change the name inside users.xml file directly both works.
>
> Last part which is my mistake. I did un-comment the legacy FileAuthorizer class at the bottom of the authorizer.xml file. I thought it will be the same procedure to do like enabling ldap-provider in the local-identity-provider.xml. I am not sure how easy other fall to this mistake.
>
> These are my main challenges over building the secured NiFi. The problem maybe would happen for person without LDAP experience like me. Otherwise there are no big problem. I haven't tried the Kerberos one which I'd love to try other time.
>
> Best Regards,
> Ken