You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by Mark Bean <ma...@gmail.com> on 2017/02/23 17:34:41 UTC
site-to-site configuration
I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
secured NiFi, and am able to access the UI securely via HTTPS. I have set
the following security-related properties:
nifi.sensitive.props.key=<key-value>
nifi.sensitive.props.key.protected=
nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
nifi.sensitive.props.provider=BC
nifi.sensitive.props.aditional.keys=
nifi.security.keystore=<keystore-file>
nifi.security.keystoreType=JKS
nifi.security.keystorePasswd=<password>
nifi.security.keyPasswd=<password>
nifi.security.truststore=<truststore-file>
nifi.security.truststoreType=JKS
nifi.security.trsustorePasswd=<password>
nifi.security.needClientAuth=true
nifi.security.user.authorizer=file-provider
nifi.security.user.login.identity.provider=
I also set the site-to-site properties:
nifi.remote.input.host=<host-fqdn>
nifi.remote.input.secure=true
nifi.remote.input.socket.port=<port, different from https UI port>
nifi.remote.input.http.enabled=true
nifi.remote.input.http.tansaction.ttl=30 sec
The authorizers.xml has been setup to import the legacy
authorized-users.xml. And, this correctly populated the users.xml to
include the remote server for the site-to-site. It also added users to the
authorizations.xml file to include the user (i.e.server ) with site-to-site
resource (both R and W).
Despite this setup, the Input Port on the UI does not show an Access
Control tab as in NiFi 0.x. I am not sure how to authorize the remote
server such that the Input Port will be displayed in the remote server's
Remote Process Group's list of ports.
Have I missed a step in the security and/or user authentication setup?
Thanks,
Mark
Re: site-to-site configuration
Posted by Andrew Lim <an...@gmail.com>.
I created a Jira to make sure we update that paragraph in the 1.x User Guide:
https://issues.apache.org/jira/browse/NIFI-3526
-Drew
> On Feb 23, 2017, at 1:48 PM, Bryan Bende <bb...@gmail.com> wrote:
>
> Mark,
>
> I think you are correct that the paragraph in the user guide should be
> updated for 1.x.
>
> I know the admin guide has a section about users and policies in
> general, but not necessarily specific to site-to-site:
>
> https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#config-users-access-policies
>
> I also have a blog post here, but I realize it is not official documentation:
>
> http://bryanbende.com/development/2016/08/30/apache-nifi-1.0.0-secure-site-to-site
>
> Thanks,
>
> Bryan
>
> On Thu, Feb 23, 2017 at 1:33 PM, Mark Bean <ma...@gmail.com> wrote:
>> Ok. Understood. I created the policy and added the user (server.) All is
>> working as expected now.
>>
>> Is this process of manipulating policies required for secure site-to-site
>> documented anywhere? The User Guide still talked about Access Control and
>> the NiFi Role which seems to apply only to 0.x.
>>
>> Thanks,
>> Mark
>>
>>
>> On Thu, Feb 23, 2017 at 1:11 PM, Bryan Bende <bb...@gmail.com> wrote:
>>
>>> Mark,
>>>
>>> When you are looking at the "receive data via site-to-site" for the
>>> input port, is there a link across the top to "Create Policy"?
>>>
>>> I think you need to create a policy first then you can add users.
>>>
>>> Thanks,
>>>
>>> Bryan
>>>
>>> On Thu, Feb 23, 2017 at 1:01 PM, Mark Bean <ma...@gmail.com> wrote:
>>>> Bryan,
>>>>
>>>> The server is listed on the global policy for "retrieve site-to-site
>>>> details". However, I am not able to add users to the "receive data via
>>>> site-to-site" policy for the given Input Port (the add user button is
>>>> grayed out.) Under global access policies, "access all policies/modify",
>>> I
>>>> am listed as a user. Shouldn't this allow me to modify the policy (i.e.
>>> add
>>>> a user) on the Input Port?
>>>>
>>>> Thanks again,
>>>> Mark
>>>>
>>>>
>>>> On Thu, Feb 23, 2017 at 12:50 PM, Bryan Bende <bb...@gmail.com> wrote:
>>>>
>>>>> Hi Mark,
>>>>>
>>>>> There are two policies needed for secure site-to-site...
>>>>>
>>>>> In the global policies there needs to be a policy for "retrieve
>>>>> site-to-site details" with the user of the server added.
>>>>>
>>>>> In the policies for the port (from the palette on the left when the
>>>>> port is selected) there needs to be a policy for "receive data via
>>>>> site-to-site" with user of the server added.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Bryan
>>>>>
>>>>> On Thu, Feb 23, 2017 at 12:34 PM, Mark Bean <ma...@gmail.com>
>>> wrote:
>>>>>> I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
>>>>>> secured NiFi, and am able to access the UI securely via HTTPS. I have
>>> set
>>>>>> the following security-related properties:
>>>>>>
>>>>>> nifi.sensitive.props.key=<key-value>
>>>>>> nifi.sensitive.props.key.protected=
>>>>>> nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
>>>>>> nifi.sensitive.props.provider=BC
>>>>>> nifi.sensitive.props.aditional.keys=
>>>>>>
>>>>>> nifi.security.keystore=<keystore-file>
>>>>>> nifi.security.keystoreType=JKS
>>>>>> nifi.security.keystorePasswd=<password>
>>>>>> nifi.security.keyPasswd=<password>
>>>>>> nifi.security.truststore=<truststore-file>
>>>>>> nifi.security.truststoreType=JKS
>>>>>> nifi.security.trsustorePasswd=<password>
>>>>>> nifi.security.needClientAuth=true
>>>>>> nifi.security.user.authorizer=file-provider
>>>>>> nifi.security.user.login.identity.provider=
>>>>>>
>>>>>> I also set the site-to-site properties:
>>>>>> nifi.remote.input.host=<host-fqdn>
>>>>>> nifi.remote.input.secure=true
>>>>>> nifi.remote.input.socket.port=<port, different from https UI port>
>>>>>> nifi.remote.input.http.enabled=true
>>>>>> nifi.remote.input.http.tansaction.ttl=30 sec
>>>>>>
>>>>>> The authorizers.xml has been setup to import the legacy
>>>>>> authorized-users.xml. And, this correctly populated the users.xml to
>>>>>> include the remote server for the site-to-site. It also added users to
>>>>> the
>>>>>> authorizations.xml file to include the user (i.e.server ) with
>>>>> site-to-site
>>>>>> resource (both R and W).
>>>>>>
>>>>>> Despite this setup, the Input Port on the UI does not show an Access
>>>>>> Control tab as in NiFi 0.x. I am not sure how to authorize the remote
>>>>>> server such that the Input Port will be displayed in the remote
>>> server's
>>>>>> Remote Process Group's list of ports.
>>>>>>
>>>>>> Have I missed a step in the security and/or user authentication setup?
>>>>>>
>>>>>> Thanks,
>>>>>> Mark
>>>>>
>>>
Re: site-to-site configuration
Posted by Bryan Bende <bb...@gmail.com>.
Mark,
I think you are correct that the paragraph in the user guide should be
updated for 1.x.
I know the admin guide has a section about users and policies in
general, but not necessarily specific to site-to-site:
https://nifi.apache.org/docs/nifi-docs/html/administration-guide.html#config-users-access-policies
I also have a blog post here, but I realize it is not official documentation:
http://bryanbende.com/development/2016/08/30/apache-nifi-1.0.0-secure-site-to-site
Thanks,
Bryan
On Thu, Feb 23, 2017 at 1:33 PM, Mark Bean <ma...@gmail.com> wrote:
> Ok. Understood. I created the policy and added the user (server.) All is
> working as expected now.
>
> Is this process of manipulating policies required for secure site-to-site
> documented anywhere? The User Guide still talked about Access Control and
> the NiFi Role which seems to apply only to 0.x.
>
> Thanks,
> Mark
>
>
> On Thu, Feb 23, 2017 at 1:11 PM, Bryan Bende <bb...@gmail.com> wrote:
>
>> Mark,
>>
>> When you are looking at the "receive data via site-to-site" for the
>> input port, is there a link across the top to "Create Policy"?
>>
>> I think you need to create a policy first then you can add users.
>>
>> Thanks,
>>
>> Bryan
>>
>> On Thu, Feb 23, 2017 at 1:01 PM, Mark Bean <ma...@gmail.com> wrote:
>> > Bryan,
>> >
>> > The server is listed on the global policy for "retrieve site-to-site
>> > details". However, I am not able to add users to the "receive data via
>> > site-to-site" policy for the given Input Port (the add user button is
>> > grayed out.) Under global access policies, "access all policies/modify",
>> I
>> > am listed as a user. Shouldn't this allow me to modify the policy (i.e.
>> add
>> > a user) on the Input Port?
>> >
>> > Thanks again,
>> > Mark
>> >
>> >
>> > On Thu, Feb 23, 2017 at 12:50 PM, Bryan Bende <bb...@gmail.com> wrote:
>> >
>> >> Hi Mark,
>> >>
>> >> There are two policies needed for secure site-to-site...
>> >>
>> >> In the global policies there needs to be a policy for "retrieve
>> >> site-to-site details" with the user of the server added.
>> >>
>> >> In the policies for the port (from the palette on the left when the
>> >> port is selected) there needs to be a policy for "receive data via
>> >> site-to-site" with user of the server added.
>> >>
>> >> Thanks,
>> >>
>> >> Bryan
>> >>
>> >> On Thu, Feb 23, 2017 at 12:34 PM, Mark Bean <ma...@gmail.com>
>> wrote:
>> >> > I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
>> >> > secured NiFi, and am able to access the UI securely via HTTPS. I have
>> set
>> >> > the following security-related properties:
>> >> >
>> >> > nifi.sensitive.props.key=<key-value>
>> >> > nifi.sensitive.props.key.protected=
>> >> > nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
>> >> > nifi.sensitive.props.provider=BC
>> >> > nifi.sensitive.props.aditional.keys=
>> >> >
>> >> > nifi.security.keystore=<keystore-file>
>> >> > nifi.security.keystoreType=JKS
>> >> > nifi.security.keystorePasswd=<password>
>> >> > nifi.security.keyPasswd=<password>
>> >> > nifi.security.truststore=<truststore-file>
>> >> > nifi.security.truststoreType=JKS
>> >> > nifi.security.trsustorePasswd=<password>
>> >> > nifi.security.needClientAuth=true
>> >> > nifi.security.user.authorizer=file-provider
>> >> > nifi.security.user.login.identity.provider=
>> >> >
>> >> > I also set the site-to-site properties:
>> >> > nifi.remote.input.host=<host-fqdn>
>> >> > nifi.remote.input.secure=true
>> >> > nifi.remote.input.socket.port=<port, different from https UI port>
>> >> > nifi.remote.input.http.enabled=true
>> >> > nifi.remote.input.http.tansaction.ttl=30 sec
>> >> >
>> >> > The authorizers.xml has been setup to import the legacy
>> >> > authorized-users.xml. And, this correctly populated the users.xml to
>> >> > include the remote server for the site-to-site. It also added users to
>> >> the
>> >> > authorizations.xml file to include the user (i.e.server ) with
>> >> site-to-site
>> >> > resource (both R and W).
>> >> >
>> >> > Despite this setup, the Input Port on the UI does not show an Access
>> >> > Control tab as in NiFi 0.x. I am not sure how to authorize the remote
>> >> > server such that the Input Port will be displayed in the remote
>> server's
>> >> > Remote Process Group's list of ports.
>> >> >
>> >> > Have I missed a step in the security and/or user authentication setup?
>> >> >
>> >> > Thanks,
>> >> > Mark
>> >>
>>
Re: site-to-site configuration
Posted by Mark Bean <ma...@gmail.com>.
Ok. Understood. I created the policy and added the user (server.) All is
working as expected now.
Is this process of manipulating policies required for secure site-to-site
documented anywhere? The User Guide still talked about Access Control and
the NiFi Role which seems to apply only to 0.x.
Thanks,
Mark
On Thu, Feb 23, 2017 at 1:11 PM, Bryan Bende <bb...@gmail.com> wrote:
> Mark,
>
> When you are looking at the "receive data via site-to-site" for the
> input port, is there a link across the top to "Create Policy"?
>
> I think you need to create a policy first then you can add users.
>
> Thanks,
>
> Bryan
>
> On Thu, Feb 23, 2017 at 1:01 PM, Mark Bean <ma...@gmail.com> wrote:
> > Bryan,
> >
> > The server is listed on the global policy for "retrieve site-to-site
> > details". However, I am not able to add users to the "receive data via
> > site-to-site" policy for the given Input Port (the add user button is
> > grayed out.) Under global access policies, "access all policies/modify",
> I
> > am listed as a user. Shouldn't this allow me to modify the policy (i.e.
> add
> > a user) on the Input Port?
> >
> > Thanks again,
> > Mark
> >
> >
> > On Thu, Feb 23, 2017 at 12:50 PM, Bryan Bende <bb...@gmail.com> wrote:
> >
> >> Hi Mark,
> >>
> >> There are two policies needed for secure site-to-site...
> >>
> >> In the global policies there needs to be a policy for "retrieve
> >> site-to-site details" with the user of the server added.
> >>
> >> In the policies for the port (from the palette on the left when the
> >> port is selected) there needs to be a policy for "receive data via
> >> site-to-site" with user of the server added.
> >>
> >> Thanks,
> >>
> >> Bryan
> >>
> >> On Thu, Feb 23, 2017 at 12:34 PM, Mark Bean <ma...@gmail.com>
> wrote:
> >> > I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
> >> > secured NiFi, and am able to access the UI securely via HTTPS. I have
> set
> >> > the following security-related properties:
> >> >
> >> > nifi.sensitive.props.key=<key-value>
> >> > nifi.sensitive.props.key.protected=
> >> > nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
> >> > nifi.sensitive.props.provider=BC
> >> > nifi.sensitive.props.aditional.keys=
> >> >
> >> > nifi.security.keystore=<keystore-file>
> >> > nifi.security.keystoreType=JKS
> >> > nifi.security.keystorePasswd=<password>
> >> > nifi.security.keyPasswd=<password>
> >> > nifi.security.truststore=<truststore-file>
> >> > nifi.security.truststoreType=JKS
> >> > nifi.security.trsustorePasswd=<password>
> >> > nifi.security.needClientAuth=true
> >> > nifi.security.user.authorizer=file-provider
> >> > nifi.security.user.login.identity.provider=
> >> >
> >> > I also set the site-to-site properties:
> >> > nifi.remote.input.host=<host-fqdn>
> >> > nifi.remote.input.secure=true
> >> > nifi.remote.input.socket.port=<port, different from https UI port>
> >> > nifi.remote.input.http.enabled=true
> >> > nifi.remote.input.http.tansaction.ttl=30 sec
> >> >
> >> > The authorizers.xml has been setup to import the legacy
> >> > authorized-users.xml. And, this correctly populated the users.xml to
> >> > include the remote server for the site-to-site. It also added users to
> >> the
> >> > authorizations.xml file to include the user (i.e.server ) with
> >> site-to-site
> >> > resource (both R and W).
> >> >
> >> > Despite this setup, the Input Port on the UI does not show an Access
> >> > Control tab as in NiFi 0.x. I am not sure how to authorize the remote
> >> > server such that the Input Port will be displayed in the remote
> server's
> >> > Remote Process Group's list of ports.
> >> >
> >> > Have I missed a step in the security and/or user authentication setup?
> >> >
> >> > Thanks,
> >> > Mark
> >>
>
Re: site-to-site configuration
Posted by Bryan Bende <bb...@gmail.com>.
Mark,
When you are looking at the "receive data via site-to-site" for the
input port, is there a link across the top to "Create Policy"?
I think you need to create a policy first then you can add users.
Thanks,
Bryan
On Thu, Feb 23, 2017 at 1:01 PM, Mark Bean <ma...@gmail.com> wrote:
> Bryan,
>
> The server is listed on the global policy for "retrieve site-to-site
> details". However, I am not able to add users to the "receive data via
> site-to-site" policy for the given Input Port (the add user button is
> grayed out.) Under global access policies, "access all policies/modify", I
> am listed as a user. Shouldn't this allow me to modify the policy (i.e. add
> a user) on the Input Port?
>
> Thanks again,
> Mark
>
>
> On Thu, Feb 23, 2017 at 12:50 PM, Bryan Bende <bb...@gmail.com> wrote:
>
>> Hi Mark,
>>
>> There are two policies needed for secure site-to-site...
>>
>> In the global policies there needs to be a policy for "retrieve
>> site-to-site details" with the user of the server added.
>>
>> In the policies for the port (from the palette on the left when the
>> port is selected) there needs to be a policy for "receive data via
>> site-to-site" with user of the server added.
>>
>> Thanks,
>>
>> Bryan
>>
>> On Thu, Feb 23, 2017 at 12:34 PM, Mark Bean <ma...@gmail.com> wrote:
>> > I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
>> > secured NiFi, and am able to access the UI securely via HTTPS. I have set
>> > the following security-related properties:
>> >
>> > nifi.sensitive.props.key=<key-value>
>> > nifi.sensitive.props.key.protected=
>> > nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
>> > nifi.sensitive.props.provider=BC
>> > nifi.sensitive.props.aditional.keys=
>> >
>> > nifi.security.keystore=<keystore-file>
>> > nifi.security.keystoreType=JKS
>> > nifi.security.keystorePasswd=<password>
>> > nifi.security.keyPasswd=<password>
>> > nifi.security.truststore=<truststore-file>
>> > nifi.security.truststoreType=JKS
>> > nifi.security.trsustorePasswd=<password>
>> > nifi.security.needClientAuth=true
>> > nifi.security.user.authorizer=file-provider
>> > nifi.security.user.login.identity.provider=
>> >
>> > I also set the site-to-site properties:
>> > nifi.remote.input.host=<host-fqdn>
>> > nifi.remote.input.secure=true
>> > nifi.remote.input.socket.port=<port, different from https UI port>
>> > nifi.remote.input.http.enabled=true
>> > nifi.remote.input.http.tansaction.ttl=30 sec
>> >
>> > The authorizers.xml has been setup to import the legacy
>> > authorized-users.xml. And, this correctly populated the users.xml to
>> > include the remote server for the site-to-site. It also added users to
>> the
>> > authorizations.xml file to include the user (i.e.server ) with
>> site-to-site
>> > resource (both R and W).
>> >
>> > Despite this setup, the Input Port on the UI does not show an Access
>> > Control tab as in NiFi 0.x. I am not sure how to authorize the remote
>> > server such that the Input Port will be displayed in the remote server's
>> > Remote Process Group's list of ports.
>> >
>> > Have I missed a step in the security and/or user authentication setup?
>> >
>> > Thanks,
>> > Mark
>>
Re: site-to-site configuration
Posted by Mark Bean <ma...@gmail.com>.
Bryan,
The server is listed on the global policy for "retrieve site-to-site
details". However, I am not able to add users to the "receive data via
site-to-site" policy for the given Input Port (the add user button is
grayed out.) Under global access policies, "access all policies/modify", I
am listed as a user. Shouldn't this allow me to modify the policy (i.e. add
a user) on the Input Port?
Thanks again,
Mark
On Thu, Feb 23, 2017 at 12:50 PM, Bryan Bende <bb...@gmail.com> wrote:
> Hi Mark,
>
> There are two policies needed for secure site-to-site...
>
> In the global policies there needs to be a policy for "retrieve
> site-to-site details" with the user of the server added.
>
> In the policies for the port (from the palette on the left when the
> port is selected) there needs to be a policy for "receive data via
> site-to-site" with user of the server added.
>
> Thanks,
>
> Bryan
>
> On Thu, Feb 23, 2017 at 12:34 PM, Mark Bean <ma...@gmail.com> wrote:
> > I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
> > secured NiFi, and am able to access the UI securely via HTTPS. I have set
> > the following security-related properties:
> >
> > nifi.sensitive.props.key=<key-value>
> > nifi.sensitive.props.key.protected=
> > nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
> > nifi.sensitive.props.provider=BC
> > nifi.sensitive.props.aditional.keys=
> >
> > nifi.security.keystore=<keystore-file>
> > nifi.security.keystoreType=JKS
> > nifi.security.keystorePasswd=<password>
> > nifi.security.keyPasswd=<password>
> > nifi.security.truststore=<truststore-file>
> > nifi.security.truststoreType=JKS
> > nifi.security.trsustorePasswd=<password>
> > nifi.security.needClientAuth=true
> > nifi.security.user.authorizer=file-provider
> > nifi.security.user.login.identity.provider=
> >
> > I also set the site-to-site properties:
> > nifi.remote.input.host=<host-fqdn>
> > nifi.remote.input.secure=true
> > nifi.remote.input.socket.port=<port, different from https UI port>
> > nifi.remote.input.http.enabled=true
> > nifi.remote.input.http.tansaction.ttl=30 sec
> >
> > The authorizers.xml has been setup to import the legacy
> > authorized-users.xml. And, this correctly populated the users.xml to
> > include the remote server for the site-to-site. It also added users to
> the
> > authorizations.xml file to include the user (i.e.server ) with
> site-to-site
> > resource (both R and W).
> >
> > Despite this setup, the Input Port on the UI does not show an Access
> > Control tab as in NiFi 0.x. I am not sure how to authorize the remote
> > server such that the Input Port will be displayed in the remote server's
> > Remote Process Group's list of ports.
> >
> > Have I missed a step in the security and/or user authentication setup?
> >
> > Thanks,
> > Mark
>
Re: site-to-site configuration
Posted by Bryan Bende <bb...@gmail.com>.
Hi Mark,
There are two policies needed for secure site-to-site...
In the global policies there needs to be a policy for "retrieve
site-to-site details" with the user of the server added.
In the policies for the port (from the palette on the left when the
port is selected) there needs to be a policy for "receive data via
site-to-site" with user of the server added.
Thanks,
Bryan
On Thu, Feb 23, 2017 at 12:34 PM, Mark Bean <ma...@gmail.com> wrote:
> I am attempting to setup secure site-to-site using NiFi 1.1.1. I have
> secured NiFi, and am able to access the UI securely via HTTPS. I have set
> the following security-related properties:
>
> nifi.sensitive.props.key=<key-value>
> nifi.sensitive.props.key.protected=
> nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
> nifi.sensitive.props.provider=BC
> nifi.sensitive.props.aditional.keys=
>
> nifi.security.keystore=<keystore-file>
> nifi.security.keystoreType=JKS
> nifi.security.keystorePasswd=<password>
> nifi.security.keyPasswd=<password>
> nifi.security.truststore=<truststore-file>
> nifi.security.truststoreType=JKS
> nifi.security.trsustorePasswd=<password>
> nifi.security.needClientAuth=true
> nifi.security.user.authorizer=file-provider
> nifi.security.user.login.identity.provider=
>
> I also set the site-to-site properties:
> nifi.remote.input.host=<host-fqdn>
> nifi.remote.input.secure=true
> nifi.remote.input.socket.port=<port, different from https UI port>
> nifi.remote.input.http.enabled=true
> nifi.remote.input.http.tansaction.ttl=30 sec
>
> The authorizers.xml has been setup to import the legacy
> authorized-users.xml. And, this correctly populated the users.xml to
> include the remote server for the site-to-site. It also added users to the
> authorizations.xml file to include the user (i.e.server ) with site-to-site
> resource (both R and W).
>
> Despite this setup, the Input Port on the UI does not show an Access
> Control tab as in NiFi 0.x. I am not sure how to authorize the remote
> server such that the Input Port will be displayed in the remote server's
> Remote Process Group's list of ports.
>
> Have I missed a step in the security and/or user authentication setup?
>
> Thanks,
> Mark