You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Nicholas Hughes <ni...@gmail.com> on 2016/11/14 15:57:31 UTC
NiFi w/ Ranger and AD
Has anyone implemented Apache NiFi with authentication against Microsoft
Active Directory and Apache Ranger for authorization (also using AD
accounts)?
The authentication works as expected and UserSync works properly in Ranger,
but I think NiFi and Ranger might not be on the same page page when it
comes to the expected username format.
I can type in my AD sAMAccountName and password at the NiFi login screen,
and authentication is successful. Additionally, Ranger is set to sync users
from AD using the sAMAccountName and that seems to work fine. However,
authorization fails with a "Unable to perform the desired action due to
insufficient permissions. Contact the system administrator." error. I
decoded the JWT from the user log, and the payload looks like:
{
"sub": "cn=Nick Hughes,ou=Users,ou=Accounts,dc=example,dc=com",
"iss": "LdapProvider",
"aud": "LdapProvider",
"preferred_username": "Nick Hughes",
"kid": 1,
"exp": 1479180675,
"iat": 1479137475
}
I suspect that authorization isn't working since the usernames in Ranger
are the short sAMAccountName (nhughes for example) while the JWT has the CN
and DN in the token. Totally guessing, so feel free to set me straight...
Anyone have any experience here? I saw some posts on the Internet regarding
Ranger with LDAP, but there may be some idiosyncrasies with AD.
Thanks!
-Nick
Re: NiFi w/ Ranger and AD
Posted by Nicholas Hughes <ni...@gmail.com>.
Bryan,
Thanks for pointing me at that setting. It did the trick for my workaround
with the DN!
-Nick
On Sun, Nov 27, 2016 at 9:37 PM, Bryan Bende <bb...@gmail.com> wrote:
> Hey Nick,
>
> I believe Ranger's user sync has an identity transform capability [1] that
> can perform certain transformations to the identity string. So in
> ranger-ugsync-site.xml you can have:
>
> <property>
> <name>ranger.usersync.mapping.username.regex.1</name>
> <value>s/regex/replacement/g</value>
> </property>
>
> I've only used this with a file as the source of users/groups, but I would
> imagine it works the same for LDAP since all the of Ranger's user sync code
> is shared regardless of the source, and you can have more than one of them
> with a unique id at the end of the name.
>
> You might be able to combine that with NiFi's identity transform [2] to
> come up with something that gives you what you are looking for.
>
> -Bryan
>
> [1] https://issues.apache.org/jira/browse/RANGER-684
> [2] https://github.com/apache/nifi/blob/master/nifi-nar-
> bundles/nifi-framework-bundle/nifi-framework/nifi-resources/
> src/main/resources/conf/nifi.properties#L152-L160
>
>
>
> On Sun, Nov 27, 2016 at 5:56 PM, Joe Witt <jo...@gmail.com> wrote:
>
>> Nick
>>
>> Thanks for documenting those findings. Will definitely help in
>> resolving it in a more consistent manner.
>>
>> As for the release of any vendor specific distributions of NiFi you'll
>> want to connect directly with that vendor for those sorts of
>> questions.
>>
>> Thanks
>> Joe
>>
>> On Sun, Nov 27, 2016 at 7:40 PM, Nicholas Hughes
>> <ni...@gmail.com> wrote:
>> > This won't matter very soon with the impending release that will fix
>> this
>> > issue, but just so it's recorded somewhere...
>> >
>> > I tried setting up UserSync in Ranger to pull in the distinguishedName
>> from
>> > Active Directory since that is what was passed by NiFi. Unfortunately,
>> there
>> > are two problems with that...
>> >
>> > First, UserSync brings the DN in from AD with capital letters in the
>> field
>> > names (CN=,OU=,DC=) while NiFi is passing them as lowercase
>> (cn=,ou=,dc=).
>> > This causes the access request to fail to match any policies due to the
>> case
>> > mismatch. I manually converted the case in the Ranger MySQL DB as a
>> > workaround.
>> >
>> > Second, any policies created which contain the DN user names (or any
>> other
>> > names containing commas) will not be able to be edited after the initial
>> > creation. This is due to how Ranger is encoding the URL when accessing
>> the
>> > Ranger API. A ticket has been created for this issue [1], but I don't
>> think
>> > it'll get much priority since there was a comment about not having
>> commas in
>> > user names.
>> >
>> > Does anybody know how long it's likely to take for the 1.1.0 release
>> (once
>> > completed) to get rolled into HDF? I'd rather not have to hack up NiFi
>> in an
>> > HDF install in order to take advantage of the fix for sAMAccountNames
>> [2]
>> > that's in that release.
>> >
>> > -Nick
>> >
>> > [1] - https://issues.apache.org/jira/browse/RANGER-1224
>> > [2] - https://issues.apache.org/jira/browse/NIFI-3020
>> >
>> > On Mon, Nov 14, 2016 at 10:59 AM, Joe Witt <jo...@gmail.com> wrote:
>> >>
>> >> Nick - there does appear to be agreement with your finding. Take a
>> >> look here https://issues.apache.org/jira/browse/NIFI-3020
>> >>
>> >> On Mon, Nov 14, 2016 at 10:57 AM, Nicholas Hughes
>> >> <ni...@gmail.com> wrote:
>> >> > Has anyone implemented Apache NiFi with authentication against
>> Microsoft
>> >> > Active Directory and Apache Ranger for authorization (also using AD
>> >> > accounts)?
>> >> >
>> >> > The authentication works as expected and UserSync works properly in
>> >> > Ranger,
>> >> > but I think NiFi and Ranger might not be on the same page page when
>> it
>> >> > comes
>> >> > to the expected username format.
>> >> >
>> >> > I can type in my AD sAMAccountName and password at the NiFi login
>> >> > screen,
>> >> > and authentication is successful. Additionally, Ranger is set to sync
>> >> > users
>> >> > from AD using the sAMAccountName and that seems to work fine.
>> However,
>> >> > authorization fails with a "Unable to perform the desired action due
>> to
>> >> > insufficient permissions. Contact the system administrator." error. I
>> >> > decoded the JWT from the user log, and the payload looks like:
>> >> >
>> >> > {
>> >> > "sub": "cn=Nick Hughes,ou=Users,ou=Accounts,dc=example,dc=com",
>> >> > "iss": "LdapProvider",
>> >> > "aud": "LdapProvider",
>> >> > "preferred_username": "Nick Hughes",
>> >> > "kid": 1,
>> >> > "exp": 1479180675,
>> >> > "iat": 1479137475
>> >> > }
>> >> >
>> >> > I suspect that authorization isn't working since the usernames in
>> Ranger
>> >> > are
>> >> > the short sAMAccountName (nhughes for example) while the JWT has the
>> CN
>> >> > and
>> >> > DN in the token. Totally guessing, so feel free to set me straight...
>> >> >
>> >> > Anyone have any experience here? I saw some posts on the Internet
>> >> > regarding
>> >> > Ranger with LDAP, but there may be some idiosyncrasies with AD.
>> >> >
>> >> > Thanks!
>> >> >
>> >> > -Nick
>> >> >
>> >
>> >
>>
>
>
Re: NiFi w/ Ranger and AD
Posted by Bryan Bende <bb...@gmail.com>.
Hey Nick,
I believe Ranger's user sync has an identity transform capability [1] that
can perform certain transformations to the identity string. So in
ranger-ugsync-site.xml you can have:
<property>
<name>ranger.usersync.mapping.username.regex.1</name>
<value>s/regex/replacement/g</value>
</property>
I've only used this with a file as the source of users/groups, but I would
imagine it works the same for LDAP since all the of Ranger's user sync code
is shared regardless of the source, and you can have more than one of them
with a unique id at the end of the name.
You might be able to combine that with NiFi's identity transform [2] to
come up with something that gives you what you are looking for.
-Bryan
[1] https://issues.apache.org/jira/browse/RANGER-684
[2]
https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-resources/src/main/resources/conf/nifi.properties#L152-L160
On Sun, Nov 27, 2016 at 5:56 PM, Joe Witt <jo...@gmail.com> wrote:
> Nick
>
> Thanks for documenting those findings. Will definitely help in
> resolving it in a more consistent manner.
>
> As for the release of any vendor specific distributions of NiFi you'll
> want to connect directly with that vendor for those sorts of
> questions.
>
> Thanks
> Joe
>
> On Sun, Nov 27, 2016 at 7:40 PM, Nicholas Hughes
> <ni...@gmail.com> wrote:
> > This won't matter very soon with the impending release that will fix this
> > issue, but just so it's recorded somewhere...
> >
> > I tried setting up UserSync in Ranger to pull in the distinguishedName
> from
> > Active Directory since that is what was passed by NiFi. Unfortunately,
> there
> > are two problems with that...
> >
> > First, UserSync brings the DN in from AD with capital letters in the
> field
> > names (CN=,OU=,DC=) while NiFi is passing them as lowercase
> (cn=,ou=,dc=).
> > This causes the access request to fail to match any policies due to the
> case
> > mismatch. I manually converted the case in the Ranger MySQL DB as a
> > workaround.
> >
> > Second, any policies created which contain the DN user names (or any
> other
> > names containing commas) will not be able to be edited after the initial
> > creation. This is due to how Ranger is encoding the URL when accessing
> the
> > Ranger API. A ticket has been created for this issue [1], but I don't
> think
> > it'll get much priority since there was a comment about not having
> commas in
> > user names.
> >
> > Does anybody know how long it's likely to take for the 1.1.0 release
> (once
> > completed) to get rolled into HDF? I'd rather not have to hack up NiFi
> in an
> > HDF install in order to take advantage of the fix for sAMAccountNames [2]
> > that's in that release.
> >
> > -Nick
> >
> > [1] - https://issues.apache.org/jira/browse/RANGER-1224
> > [2] - https://issues.apache.org/jira/browse/NIFI-3020
> >
> > On Mon, Nov 14, 2016 at 10:59 AM, Joe Witt <jo...@gmail.com> wrote:
> >>
> >> Nick - there does appear to be agreement with your finding. Take a
> >> look here https://issues.apache.org/jira/browse/NIFI-3020
> >>
> >> On Mon, Nov 14, 2016 at 10:57 AM, Nicholas Hughes
> >> <ni...@gmail.com> wrote:
> >> > Has anyone implemented Apache NiFi with authentication against
> Microsoft
> >> > Active Directory and Apache Ranger for authorization (also using AD
> >> > accounts)?
> >> >
> >> > The authentication works as expected and UserSync works properly in
> >> > Ranger,
> >> > but I think NiFi and Ranger might not be on the same page page when it
> >> > comes
> >> > to the expected username format.
> >> >
> >> > I can type in my AD sAMAccountName and password at the NiFi login
> >> > screen,
> >> > and authentication is successful. Additionally, Ranger is set to sync
> >> > users
> >> > from AD using the sAMAccountName and that seems to work fine. However,
> >> > authorization fails with a "Unable to perform the desired action due
> to
> >> > insufficient permissions. Contact the system administrator." error. I
> >> > decoded the JWT from the user log, and the payload looks like:
> >> >
> >> > {
> >> > "sub": "cn=Nick Hughes,ou=Users,ou=Accounts,dc=example,dc=com",
> >> > "iss": "LdapProvider",
> >> > "aud": "LdapProvider",
> >> > "preferred_username": "Nick Hughes",
> >> > "kid": 1,
> >> > "exp": 1479180675,
> >> > "iat": 1479137475
> >> > }
> >> >
> >> > I suspect that authorization isn't working since the usernames in
> Ranger
> >> > are
> >> > the short sAMAccountName (nhughes for example) while the JWT has the
> CN
> >> > and
> >> > DN in the token. Totally guessing, so feel free to set me straight...
> >> >
> >> > Anyone have any experience here? I saw some posts on the Internet
> >> > regarding
> >> > Ranger with LDAP, but there may be some idiosyncrasies with AD.
> >> >
> >> > Thanks!
> >> >
> >> > -Nick
> >> >
> >
> >
>
Re: NiFi w/ Ranger and AD
Posted by Joe Witt <jo...@gmail.com>.
Nick
Thanks for documenting those findings. Will definitely help in
resolving it in a more consistent manner.
As for the release of any vendor specific distributions of NiFi you'll
want to connect directly with that vendor for those sorts of
questions.
Thanks
Joe
On Sun, Nov 27, 2016 at 7:40 PM, Nicholas Hughes
<ni...@gmail.com> wrote:
> This won't matter very soon with the impending release that will fix this
> issue, but just so it's recorded somewhere...
>
> I tried setting up UserSync in Ranger to pull in the distinguishedName from
> Active Directory since that is what was passed by NiFi. Unfortunately, there
> are two problems with that...
>
> First, UserSync brings the DN in from AD with capital letters in the field
> names (CN=,OU=,DC=) while NiFi is passing them as lowercase (cn=,ou=,dc=).
> This causes the access request to fail to match any policies due to the case
> mismatch. I manually converted the case in the Ranger MySQL DB as a
> workaround.
>
> Second, any policies created which contain the DN user names (or any other
> names containing commas) will not be able to be edited after the initial
> creation. This is due to how Ranger is encoding the URL when accessing the
> Ranger API. A ticket has been created for this issue [1], but I don't think
> it'll get much priority since there was a comment about not having commas in
> user names.
>
> Does anybody know how long it's likely to take for the 1.1.0 release (once
> completed) to get rolled into HDF? I'd rather not have to hack up NiFi in an
> HDF install in order to take advantage of the fix for sAMAccountNames [2]
> that's in that release.
>
> -Nick
>
> [1] - https://issues.apache.org/jira/browse/RANGER-1224
> [2] - https://issues.apache.org/jira/browse/NIFI-3020
>
> On Mon, Nov 14, 2016 at 10:59 AM, Joe Witt <jo...@gmail.com> wrote:
>>
>> Nick - there does appear to be agreement with your finding. Take a
>> look here https://issues.apache.org/jira/browse/NIFI-3020
>>
>> On Mon, Nov 14, 2016 at 10:57 AM, Nicholas Hughes
>> <ni...@gmail.com> wrote:
>> > Has anyone implemented Apache NiFi with authentication against Microsoft
>> > Active Directory and Apache Ranger for authorization (also using AD
>> > accounts)?
>> >
>> > The authentication works as expected and UserSync works properly in
>> > Ranger,
>> > but I think NiFi and Ranger might not be on the same page page when it
>> > comes
>> > to the expected username format.
>> >
>> > I can type in my AD sAMAccountName and password at the NiFi login
>> > screen,
>> > and authentication is successful. Additionally, Ranger is set to sync
>> > users
>> > from AD using the sAMAccountName and that seems to work fine. However,
>> > authorization fails with a "Unable to perform the desired action due to
>> > insufficient permissions. Contact the system administrator." error. I
>> > decoded the JWT from the user log, and the payload looks like:
>> >
>> > {
>> > "sub": "cn=Nick Hughes,ou=Users,ou=Accounts,dc=example,dc=com",
>> > "iss": "LdapProvider",
>> > "aud": "LdapProvider",
>> > "preferred_username": "Nick Hughes",
>> > "kid": 1,
>> > "exp": 1479180675,
>> > "iat": 1479137475
>> > }
>> >
>> > I suspect that authorization isn't working since the usernames in Ranger
>> > are
>> > the short sAMAccountName (nhughes for example) while the JWT has the CN
>> > and
>> > DN in the token. Totally guessing, so feel free to set me straight...
>> >
>> > Anyone have any experience here? I saw some posts on the Internet
>> > regarding
>> > Ranger with LDAP, but there may be some idiosyncrasies with AD.
>> >
>> > Thanks!
>> >
>> > -Nick
>> >
>
>
Re: NiFi w/ Ranger and AD
Posted by Nicholas Hughes <ni...@gmail.com>.
This won't matter very soon with the impending release that will fix this
issue, but just so it's recorded somewhere...
I tried setting up UserSync in Ranger to pull in the distinguishedName from
Active Directory since that is what was passed by NiFi. Unfortunately,
there are two problems with that...
First, UserSync brings the DN in from AD with capital letters in the field
names (CN=,OU=,DC=) while NiFi is passing them as lowercase (cn=,ou=,dc=).
This causes the access request to fail to match any policies due to the
case mismatch. I manually converted the case in the Ranger MySQL DB as a
workaround.
Second, any policies created which contain the DN user names (or any other
names containing commas) will not be able to be edited after the initial
creation. This is due to how Ranger is encoding the URL when accessing the
Ranger API. A ticket has been created for this issue [1], but I don't think
it'll get much priority since there was a comment about not having commas
in user names.
Does anybody know how long it's likely to take for the 1.1.0 release (once
completed) to get rolled into HDF? I'd rather not have to hack up NiFi in
an HDF install in order to take advantage of the fix for sAMAccountNames
[2] that's in that release.
-Nick
[1] - https://issues.apache.org/jira/browse/RANGER-1224
[2] - https://issues.apache.org/jira/browse/NIFI-3020
On Mon, Nov 14, 2016 at 10:59 AM, Joe Witt <jo...@gmail.com> wrote:
> Nick - there does appear to be agreement with your finding. Take a
> look here https://issues.apache.org/jira/browse/NIFI-3020
>
> On Mon, Nov 14, 2016 at 10:57 AM, Nicholas Hughes
> <ni...@gmail.com> wrote:
> > Has anyone implemented Apache NiFi with authentication against Microsoft
> > Active Directory and Apache Ranger for authorization (also using AD
> > accounts)?
> >
> > The authentication works as expected and UserSync works properly in
> Ranger,
> > but I think NiFi and Ranger might not be on the same page page when it
> comes
> > to the expected username format.
> >
> > I can type in my AD sAMAccountName and password at the NiFi login screen,
> > and authentication is successful. Additionally, Ranger is set to sync
> users
> > from AD using the sAMAccountName and that seems to work fine. However,
> > authorization fails with a "Unable to perform the desired action due to
> > insufficient permissions. Contact the system administrator." error. I
> > decoded the JWT from the user log, and the payload looks like:
> >
> > {
> > "sub": "cn=Nick Hughes,ou=Users,ou=Accounts,dc=example,dc=com",
> > "iss": "LdapProvider",
> > "aud": "LdapProvider",
> > "preferred_username": "Nick Hughes",
> > "kid": 1,
> > "exp": 1479180675,
> > "iat": 1479137475
> > }
> >
> > I suspect that authorization isn't working since the usernames in Ranger
> are
> > the short sAMAccountName (nhughes for example) while the JWT has the CN
> and
> > DN in the token. Totally guessing, so feel free to set me straight...
> >
> > Anyone have any experience here? I saw some posts on the Internet
> regarding
> > Ranger with LDAP, but there may be some idiosyncrasies with AD.
> >
> > Thanks!
> >
> > -Nick
> >
>
Re: NiFi w/ Ranger and AD
Posted by Joe Witt <jo...@gmail.com>.
Nick - there does appear to be agreement with your finding. Take a
look here https://issues.apache.org/jira/browse/NIFI-3020
On Mon, Nov 14, 2016 at 10:57 AM, Nicholas Hughes
<ni...@gmail.com> wrote:
> Has anyone implemented Apache NiFi with authentication against Microsoft
> Active Directory and Apache Ranger for authorization (also using AD
> accounts)?
>
> The authentication works as expected and UserSync works properly in Ranger,
> but I think NiFi and Ranger might not be on the same page page when it comes
> to the expected username format.
>
> I can type in my AD sAMAccountName and password at the NiFi login screen,
> and authentication is successful. Additionally, Ranger is set to sync users
> from AD using the sAMAccountName and that seems to work fine. However,
> authorization fails with a "Unable to perform the desired action due to
> insufficient permissions. Contact the system administrator." error. I
> decoded the JWT from the user log, and the payload looks like:
>
> {
> "sub": "cn=Nick Hughes,ou=Users,ou=Accounts,dc=example,dc=com",
> "iss": "LdapProvider",
> "aud": "LdapProvider",
> "preferred_username": "Nick Hughes",
> "kid": 1,
> "exp": 1479180675,
> "iat": 1479137475
> }
>
> I suspect that authorization isn't working since the usernames in Ranger are
> the short sAMAccountName (nhughes for example) while the JWT has the CN and
> DN in the token. Totally guessing, so feel free to set me straight...
>
> Anyone have any experience here? I saw some posts on the Internet regarding
> Ranger with LDAP, but there may be some idiosyncrasies with AD.
>
> Thanks!
>
> -Nick
>