You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by nifi-san <na...@gmail.com> on 2017/07/24 10:46:56 UTC
Nifi Site-to-Site pre-existing deployments do not work after
enabling TLS and Ldap
Hello Team,
We have two environments of Nifi ,one whihc is a standalone and the other
whihc is a cluster.
I have upgraded the Nifi (standalone as well as cluster) in our non prod
environment from 1.1.1 to 1.3.0 ,Implement TLS and also integrate with LDAP.
I followed the process mentioned in the documentation and infact had a
complete parallel set up for Nifi-1.3.0 with its own repositories and
configuration.
In other words, the high level steps followed were:-
Installed Nifi-1.3.0 on a different path.
Installed toolkit and generated all the certificates.
Made all the necessary changes in the nifi.properties files on the
Nifi-1.3.0 cluster for SSL.
Stopped the old cluster and copied over the repositories and the
authorizers.xml file.Added the Initial DN and the Initial Identity to the
authorizers.xml file.
Started the new Nifi-1.3.0 cluster and logged in using the Initial Admin and
created the users specific to each of the node DN's
Imported the client certificate onto the Browser and logged into the UI.
Made the necessary confguration to include LDAP integration.Created all the
users in LDAP within Nifi (since there is no way to sync Ldap and Nifi user
list).
Post this,I was able to login into the UI of Nifi using the username and
password and get the authentication/authorization done through Nifi
successfully.
I tried doing a new site -to site deployment which worked successfully.
Source :- GetFile ->(Using IP1 ) ->RPG
Destination :- Input Port(IP1) -PutFile
For this to work,I ensured that all the users were added to the policy
"Retrieve Site-to-Site" on the destination node.Also, enabled "Receive
Site-to-Site" policy on the Input Port on the destination IP1.
However , when I take a look at the previously present Site-to-Site
deployments that existed prior to TLS and LDAP ,I see that the input ports
do not show up the policy "Receive Site-to-Site" as it is grayed out.
We are in the process of performing this in production and have the below
concerns:-
1)What will happen to the Site-to-Site deployments that existed prior to
securing the cluster and integration with LDAP?We do not have any user
authentication on the cluster in Prod right now.For site-to-site deployment
to work, we need to enable the policy on the input port "Receive
Site-to-Site".Will the pre-existing site-to-site deployment start failing?
2)How can we get the pre-existing site-to-site deployment to work as I can
see that the policy "Receive Site-to-Site" deployment is grayed out?
Appreciate any inputs!
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Site-to-Site-pre-existing-deployments-do-not-work-after-enabling-TLS-and-Ldap-tp16486.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Nifi Site-to-Site pre-existing deployments do not work after
enabling TLS and Ldap
Posted by Mark Payne <ma...@hotmail.com>.
Hello,
Yes, that is true. If you change the web port of the NiFi instance, it would be considered a 'new instance' in terms
of Remote Process Groups. So any NiFi instance that is sending to/pulling from that instance would have to create
a new Remote Process Group to point to the new URL. This probably should be improved by allowing the user to
change the URL that the Remote Process Group points to. Feel free to file a JIRA [1] if you think this would be a
helpful improvement.
In terms of updating policies, yes you would need to give permissions to all of the ports that you want users to have
access to. Typically, this is best done by using Groups so that each port can be configured to allow Site-to-Site for a
Group of users and then you can just change the members of that Group. This way you don't have to update many
Ports each time that you want to change permissions. Do you have other ideas in mind of how this would be made
easier?
Thanks
-Mark
[1] https://issues.apache.org/jira/projects/NIFI
On Aug 2, 2017, at 6:45 AM, nifi-san <na...@gmail.com>> wrote:
Thanks Mark,
I had a follow up question though.
Let's say you have a nifi flow with site-to-site deployment between two
nodes node-1- source and node-2
destination on a non secure cluster.
The default http port "8080" is used in the configuration of the RPG on node
which is http://node-2:8080/nifi.
Once you configure ssl to secure your cluster,you may have the node
bootstrapping on the https port ,let's say 9966.
All the previously configured RPG's will get affected because of this since
the nodes are no longer going to listen on the http port.
How do we handle such a scenario?You cannot even manually modify an existing
RPC to listen to the new ssl port.
Also, with the ssl configuration, on the remote node where the Input Port is
configured ,you would need to modify the access policies to "Receive data
site-to-site".
Assuming you have quite a few flows using RPG, manually changing them might
be very difficult.
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Site-to-Site-pre-existing-deployments-do-not-work-after-enabling-TLS-and-Ldap-tp16486p16560.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com<http://Nabble.com>.
Re: Nifi Site-to-Site pre-existing deployments do not work after
enabling TLS and Ldap
Posted by nifi-san <na...@gmail.com>.
Thanks Mark,
I had a follow up question though.
Let's say you have a nifi flow with site-to-site deployment between two
nodes node-1- source and node-2
destination on a non secure cluster.
The default http port "8080" is used in the configuration of the RPG on node
which is http://node-2:8080/nifi.
Once you configure ssl to secure your cluster,you may have the node
bootstrapping on the https port ,let's say 9966.
All the previously configured RPG's will get affected because of this since
the nodes are no longer going to listen on the http port.
How do we handle such a scenario?You cannot even manually modify an existing
RPC to listen to the new ssl port.
Also, with the ssl configuration, on the remote node where the Input Port is
configured ,you would need to modify the access policies to "Receive data
site-to-site".
Assuming you have quite a few flows using RPG, manually changing them might
be very difficult.
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Site-to-Site-pre-existing-deployments-do-not-work-after-enabling-TLS-and-Ldap-tp16486p16560.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Nifi Site-to-Site pre-existing deployments do not work after
enabling TLS and Ldap
Posted by nifi-san <na...@gmail.com>.
Thanks Mark.
That really helps.Appreciate it!
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Site-to-Site-pre-existing-deployments-do-not-work-after-enabling-TLS-and-Ldap-tp16486p16493.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Nifi Site-to-Site pre-existing deployments do not work after
enabling TLS and Ldap
Posted by Mark Payne <ma...@hotmail.com>.
Hello,
Site-to-Site is a feature that is only available for the Input Ports and Output Ports on the Root Process Group.
I.e., you cannot enable site-to-site on Input Ports and Output Ports within a child Process Group.
The function of an Input Port or an Output Port in NiFi is to get data into or out of a Process Group.
So as data flows through your flow, it can enter a Process Group via its Input Port(s) and when it's
finished the Process Group, it will exit back up to the parent Process Group's components via the
Output Port.
The idea behind site-to-site was to treat a NiFi instance's Root-level Process Group like other Process Groups,
by allowing you to send data into them using Input Ports/Output Ports. Then you simply send to the Root-level
Process Group from another instance by using a Remote Process Group.
So typically if you want to push data into a child Process Group, you will create an Input Port on the Root-level
Process Group, and then connect that Root-level Input Port to your child Process Group's Input Port.
Does that make sense?
Thanks
-Mark
> On Jul 24, 2017, at 8:07 AM, nifi-san <na...@gmail.com> wrote:
>
> Basically, on the destination Nifi cluster,at a Global Policy level,I have
> enabled "Retrieve Site-to-Site" for the source nifi user.
>
> On the default root Nifi-flow processor,on the input port,I can see the
> policy "Receive site-to-Site"
>
> However,when I create a new Process Group ,let's say Test and then try to
> create an Input Port ,I see that the option "Receive Site-to-Site" is grayed
> out.
>
> What is a user already has site-to-site deployment on a non default process
> group such as test prior to configuring authentication and authorization?
>
>
>
>
>
>
>
>
>
>
> --
> View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Site-to-Site-pre-existing-deployments-do-not-work-after-enabling-TLS-and-Ldap-tp16486p16488.html
> Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.
Re: Nifi Site-to-Site pre-existing deployments do not work after
enabling TLS and Ldap
Posted by nifi-san <na...@gmail.com>.
Basically, on the destination Nifi cluster,at a Global Policy level,I have
enabled "Retrieve Site-to-Site" for the source nifi user.
On the default root Nifi-flow processor,on the input port,I can see the
policy "Receive site-to-Site"
However,when I create a new Process Group ,let's say Test and then try to
create an Input Port ,I see that the option "Receive Site-to-Site" is grayed
out.
What is a user already has site-to-site deployment on a non default process
group such as test prior to configuring authentication and authorization?
--
View this message in context: http://apache-nifi-developer-list.39713.n7.nabble.com/Nifi-Site-to-Site-pre-existing-deployments-do-not-work-after-enabling-TLS-and-Ldap-tp16486p16488.html
Sent from the Apache NiFi Developer List mailing list archive at Nabble.com.