You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nifi.apache.org by James McMahon <js...@gmail.com> on 2018/05/29 22:54:32 UTC
User, Group in LDAP appear to be unknown to PutFile
Good evening. I have recently migrated my nifi service host server from
local resolution of users and groups to use an LDAP server. I configured
login-identity-providers.xml and
nifi.security.user.login.identity.provider. I verified my configuration is
known to NiFi by first restarting my nifi service and then attempting a
login to the URL by a user without a cert, forcing it to resolve using
LDAP. This appeared to work.
I then attempted to set my file owner and file group in a PutFile to a user
and a group that are each in the LDAP. The PutFile throws a Warning for
both owner and group:
java.nio.file.attribute.UserPrincipalNotFoundException. The file is still
output by the processor. It appears to default the user and owner to nifi.
A cursory review of the PutFile source shows that PutFile employs
getUserPrincipalLookupService() when it seemingly tries to validate the
user and group.
How can I get this to resolve through the LDAP for the PutFile?
Thanks for any insights. -Jim
Re: User, Group in LDAP appear to be unknown to PutFile
Posted by James McMahon <js...@gmail.com>.
My apologies. It is sometimes difficult to decide if the root cause of a
challenge is related to code or related to a user level configuration
issue. In this case I had included the developer group because it seemed it
might relate to the PutFile after I had eliminated the possible
configuration omissions.
On Tue, May 29, 2018 at 7:14 PM, Joe Witt <jo...@gmail.com> wrote:
> jim
>
> please only post to one list.
>
> users is good for this.
>
> thanks
> joe
>
> On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com> wrote:
>
>> Good evening. I have recently migrated my nifi service host server from
>> local resolution of users and groups to use an LDAP server. I configured
>> login-identity-providers.xml and nifi.security.user.login.identity.provider.
>> I verified my configuration is known to NiFi by first restarting my nifi
>> service and then attempting a login to the URL by a user without a cert,
>> forcing it to resolve using LDAP. This appeared to work.
>>
>> I then attempted to set my file owner and file group in a PutFile to a
>> user and a group that are each in the LDAP. The PutFile throws a Warning
>> for both owner and group: java.nio.file.attribute.
>> UserPrincipalNotFoundException. The file is still output by the
>> processor. It appears to default the user and owner to nifi.
>>
>> A cursory review of the PutFile source shows that PutFile employs
>> getUserPrincipalLookupService() when it seemingly tries to validate the
>> user and group.
>>
>> How can I get this to resolve through the LDAP for the PutFile?
>>
>> Thanks for any insights. -Jim
>>
>
RE: User, Group in LDAP appear to be unknown to PutFile
Posted by Shawn Weeks <sw...@weeksconsulting.us>.
We’re using SSSD not LDAP directly so ours look like this. You can use LDAP as the source for SSSD but I’m not exactly sure why the behavior would be different.
passwd: files sss
shadow: files sss
group: files sss
I’m looking back through the thread and I haven’t seen it mentioned, are you just using LDAP or is Kerberos in the mix, we’re using an integrated system with FreeIPA which might make some differences.
Can you post the exact error and jdk version? I’ll go take a look at the Oracle source to see exactly what Java is trying to do.
Thanks
Shawn
From: James McMahon <js...@gmail.com>
Sent: Wednesday, May 30, 2018 1:41 PM
To: users@nifi.apache.org
Subject: Re: User, Group in LDAP appear to be unknown to PutFile
Yes sir - we are indeed able to create files with that group. By chance, are you using /etc/nsswitch.conf? Do your entries for passwd, shadow, and group look something like this?
passwd: files ldap
shadow: files ldap
group: files ldap
(We did try to reverse that order to "ldap files". The same warnings get thrown).
On Wed, May 30, 2018 at 11:11 AM, Shawn Weeks <sw...@weeksconsulting.us>> wrote:
Logged in as the user NiFi is running as on the same host are you able to create files with that group? We use PutFile and none of our groups are local to the host.
Thanks
Shawn
From: James McMahon <js...@gmail.com>>
Sent: Wednesday, May 30, 2018 8:21 AM
To: users@nifi.apache.org<ma...@nifi.apache.org>
Subject: Re: User, Group in LDAP appear to be unknown to PutFile
I did indeed configure PutFile as follows:
Permissions ..... 775
Owner ..... nifi
Group ..... ext_dev
When nifi is in local /etc/passwd and ext_dev is in local /etc/group, the PutFile succeeds.
When neither exists in the local files, I get the Warning in both case and the file is output with nifi:nifi (the default, as you mentioned Pierre).
When one is put back in the local file and the other left in the ldap, the one in the local succeeds and the one only in the ldap still throws the Warning and defaults on the output file.
We have also tried to reorder in our nsswitch.conf. For instance, changing
group: files ldap
to
group: ldap files
This did not change the results.
From the NiFi processor, user and group resolution do not appear to be looking beyond a check in the local files.
I did double check that user nifi exists only in either the local file or the ldap. Likewise for group ext_dev.
On Wed, May 30, 2018 at 8:52 AM, Pierre Villard <pi...@gmail.com>> wrote:
By default, PutFile will set the ownership of the file to the user running the NiFi instance (nifi if NiFi is running as nifi user). Then, if you configured a different ownership in the processor configuration it'll try to set the ownership using the username you configured in the processor. What did you set in the processor parameters for owner/group and permission?
2018-05-30 14:39 GMT+02:00 James McMahon <js...@gmail.com>>:
Yes sir, sure does. In this instance my user nifi does indeed resolve at the OS level - I think that gives us some confidence it does resolve. The lookupPrincipalByName(owner) within the PutFile is where I believe the failure is rooted, but I do not understand how that function executes its lookup. I'm trying to dig deeper into that now.
On Wed, May 30, 2018 at 8:30 AM, Pierre Villard <pi...@gmail.com>> wrote:
I think we're saying the same :) Let me rephrase it differently: to set the owner of a file, the user needs to be resolved at OS level. If the user does not exist (from the OS point of view), NiFi won't be able to set the owner (even though the username is in the LDAP configured for NiFi authentication). Does it make more sense?
$ touch test.file
$ id toto
id: toto: no such user
$ chown toto test.file
chown: toto: illegal user name
Pierre
2018-05-30 14:13 GMT+02:00 James McMahon <js...@gmail.com>>:
I don't understand this:
"Until you can't resolve the user with OS commands, I don't think NiFi will be able to set the expected owner on the file"
Did you intend to say can there - don't we want to be able to resolve the user at the OS as an initial validation that we can get to the ldap and as prerequisite of Nifi being able to set the owner? And in this case I can indeed resolve at the OS level. That seems like a good thing to me.
On Wed, May 30, 2018 at 7:53 AM, Pierre Villard <pi...@gmail.com>> wrote:
It depends how your OS is configured, you could leverage tools like SSSD to resolve users against your LDAP but that's something to be configured at OS level.
Until you can't resolve the user with OS commands, I don't think NiFi will be able to set the expected owner on the file.
2018-05-30 11:54 GMT+02:00 James McMahon <js...@gmail.com>>:
Hello Pierre, and thank you. The user in this case - nifi - is not in the local /etc/passwd and is in the ldap. I presume this will force the id <username> to resolve using the ldap, if it does resolve? At the OS the id command returns the uid, the gid, and the groups to which user nifi has membership within the ldap.
On Wed, May 30, 2018 at 4:37 AM, Pierre Villard <pi...@gmail.com>> wrote:
Hi Jim,
LDAP for authentication and authorizations in NiFi has nothing to do with the processors.
How processors are running/working is completely independent to the authN/authZ model you configure for NiFi.
Regarding your error, I'd say that you get this error because user/group you're setting in the processor configuration cannot be resolved at OS level (even though they exist in the LDAP, but again, that's totally unrelated). Something you can quickly check: can you resolve the username/group on the host where you're using PutFile processor? What do you get if you execute the following command: id <username>?
Pierre
2018-05-30 1:14 GMT+02:00 Joe Witt <jo...@gmail.com>>:
jim
please only post to one list.
users is good for this.
thanks
joe
On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com>> wrote:
Good evening. I have recently migrated my nifi service host server from local resolution of users and groups to use an LDAP server. I configured login-identity-providers.xml and nifi.security.user.login.identity.provider. I verified my configuration is known to NiFi by first restarting my nifi service and then attempting a login to the URL by a user without a cert, forcing it to resolve using LDAP. This appeared to work.
I then attempted to set my file owner and file group in a PutFile to a user and a group that are each in the LDAP. The PutFile throws a Warning for both owner and group: java.nio.file.attribute.UserPrincipalNotFoundException. The file is still output by the processor. It appears to default the user and owner to nifi.
A cursory review of the PutFile source shows that PutFile employs getUserPrincipalLookupService() when it seemingly tries to validate the user and group.
How can I get this to resolve through the LDAP for the PutFile?
Thanks for any insights. -Jim
Re: User, Group in LDAP appear to be unknown to PutFile
Posted by James McMahon <js...@gmail.com>.
Yes sir - we are indeed able to create files with that group. By chance,
are you using /etc/nsswitch.conf? Do your entries for passwd, shadow, and
group look something like this?
passwd: files ldap
shadow: files ldap
group: files ldap
(We did try to reverse that order to "ldap files". The same warnings get
thrown).
On Wed, May 30, 2018 at 11:11 AM, Shawn Weeks <sw...@weeksconsulting.us>
wrote:
> Logged in as the user NiFi is running as on the same host are you able to
> create files with that group? We use PutFile and none of our groups are
> local to the host.
>
>
>
> Thanks
>
> Shawn
>
>
>
> *From:* James McMahon <js...@gmail.com>
> *Sent:* Wednesday, May 30, 2018 8:21 AM
> *To:* users@nifi.apache.org
> *Subject:* Re: User, Group in LDAP appear to be unknown to PutFile
>
>
>
> I did indeed configure PutFile as follows:
>
> Permissions ..... 775
>
> Owner ..... nifi
>
> Group ..... ext_dev
>
> When nifi is in local /etc/passwd and ext_dev is in local /etc/group, the
> PutFile succeeds.
>
> When neither exists in the local files, I get the Warning in both case and
> the file is output with nifi:nifi (the default, as you mentioned Pierre).
>
> When one is put back in the local file and the other left in the ldap, the
> one in the local succeeds and the one only in the ldap still throws the
> Warning and defaults on the output file.
>
> We have also tried to reorder in our nsswitch.conf. For instance, changing
>
> group: files ldap
>
> to
>
> group: ldap files
>
> This did not change the results.
>
>
>
> From the NiFi processor, user and group resolution do not appear to be
> looking beyond a check in the local files.
>
> I did double check that user nifi exists only in either the local file or
> the ldap. Likewise for group ext_dev.
>
>
>
>
>
> On Wed, May 30, 2018 at 8:52 AM, Pierre Villard <
> pierre.villard.fr@gmail.com> wrote:
>
> By default, PutFile will set the ownership of the file to the user running
> the NiFi instance (nifi if NiFi is running as nifi user). Then, if you
> configured a different ownership in the processor configuration it'll try
> to set the ownership using the username you configured in the processor.
> What did you set in the processor parameters for owner/group and permission?
>
>
>
> 2018-05-30 14:39 GMT+02:00 James McMahon <js...@gmail.com>:
>
> Yes sir, sure does. In this instance my user nifi does indeed resolve at
> the OS level - I think that gives us some confidence it does resolve. The
> lookupPrincipalByName(owner) within the PutFile is where I believe the
> failure is rooted, but I do not understand how that function executes its
> lookup. I'm trying to dig deeper into that now.
>
>
>
> On Wed, May 30, 2018 at 8:30 AM, Pierre Villard <
> pierre.villard.fr@gmail.com> wrote:
>
> I think we're saying the same :) Let me rephrase it differently: to set
> the owner of a file, the user needs to be resolved at OS level. If the user
> does not exist (from the OS point of view), NiFi won't be able to set the
> owner (even though the username is in the LDAP configured for NiFi
> authentication). Does it make more sense?
>
>
>
> $ touch test.file
> $ id toto
>
> id: toto: no such user
>
> $ chown toto test.file
>
> chown: toto: illegal user name
>
>
>
> Pierre
>
>
>
>
>
>
>
>
>
> 2018-05-30 14:13 GMT+02:00 James McMahon <js...@gmail.com>:
>
> I don't understand this:
> "Until you *can't* resolve the user with OS commands, I don't think NiFi
> will be able to set the expected owner on the file"
>
> Did you intend to say can there - don't we want to be able to resolve the
> user at the OS as an initial validation that we can get to the ldap and as
> prerequisite of Nifi being able to set the owner? And in this case I can
> indeed resolve at the OS level. That seems like a good thing to me.
>
>
>
> On Wed, May 30, 2018 at 7:53 AM, Pierre Villard <
> pierre.villard.fr@gmail.com> wrote:
>
> It depends how your OS is configured, you could leverage tools like SSSD
> to resolve users against your LDAP but that's something to be configured at
> OS level.
>
> Until you can't resolve the user with OS commands, I don't think NiFi will
> be able to set the expected owner on the file.
>
>
>
> 2018-05-30 11:54 GMT+02:00 James McMahon <js...@gmail.com>:
>
> Hello Pierre, and thank you. The user in this case - nifi - is not in the
> local /etc/passwd and is in the ldap. I presume this will force the id
> <username> to resolve using the ldap, if it does resolve? At the OS the id
> command returns the uid, the gid, and the groups to which user nifi has
> membership within the ldap.
>
>
>
> On Wed, May 30, 2018 at 4:37 AM, Pierre Villard <
> pierre.villard.fr@gmail.com> wrote:
>
> Hi Jim,
>
>
>
> LDAP for authentication and authorizations in NiFi has nothing to do with
> the processors.
>
> How processors are running/working is completely independent to the
> authN/authZ model you configure for NiFi.
>
>
>
> Regarding your error, I'd say that you get this error because user/group
> you're setting in the processor configuration cannot be resolved at OS
> level (even though they exist in the LDAP, but again, that's totally
> unrelated). Something you can quickly check: can you resolve the
> username/group on the host where you're using PutFile processor? What do
> you get if you execute the following command: id <username>?
>
>
>
> Pierre
>
>
>
> 2018-05-30 1:14 GMT+02:00 Joe Witt <jo...@gmail.com>:
>
> jim
>
>
>
> please only post to one list.
>
>
> users is good for this.
>
>
>
> thanks
>
> joe
>
>
>
> On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com> wrote:
>
> Good evening. I have recently migrated my nifi service host server from
> local resolution of users and groups to use an LDAP server. I configured
> login-identity-providers.xml and nifi.security.user.login.identity.provider.
> I verified my configuration is known to NiFi by first restarting my nifi
> service and then attempting a login to the URL by a user without a cert,
> forcing it to resolve using LDAP. This appeared to work.
>
> I then attempted to set my file owner and file group in a PutFile to a
> user and a group that are each in the LDAP. The PutFile throws a Warning
> for both owner and group: java.nio.file.attribute.
> UserPrincipalNotFoundException. The file is still output by the
> processor. It appears to default the user and owner to nifi.
>
> A cursory review of the PutFile source shows that PutFile employs
> getUserPrincipalLookupService() when it seemingly tries to validate the
> user and group.
>
> How can I get this to resolve through the LDAP for the PutFile?
>
> Thanks for any insights. -Jim
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
RE: User, Group in LDAP appear to be unknown to PutFile
Posted by Shawn Weeks <sw...@weeksconsulting.us>.
Logged in as the user NiFi is running as on the same host are you able to create files with that group? We use PutFile and none of our groups are local to the host.
Thanks
Shawn
From: James McMahon <js...@gmail.com>
Sent: Wednesday, May 30, 2018 8:21 AM
To: users@nifi.apache.org
Subject: Re: User, Group in LDAP appear to be unknown to PutFile
I did indeed configure PutFile as follows:
Permissions ..... 775
Owner ..... nifi
Group ..... ext_dev
When nifi is in local /etc/passwd and ext_dev is in local /etc/group, the PutFile succeeds.
When neither exists in the local files, I get the Warning in both case and the file is output with nifi:nifi (the default, as you mentioned Pierre).
When one is put back in the local file and the other left in the ldap, the one in the local succeeds and the one only in the ldap still throws the Warning and defaults on the output file.
We have also tried to reorder in our nsswitch.conf. For instance, changing
group: files ldap
to
group: ldap files
This did not change the results.
From the NiFi processor, user and group resolution do not appear to be looking beyond a check in the local files.
I did double check that user nifi exists only in either the local file or the ldap. Likewise for group ext_dev.
On Wed, May 30, 2018 at 8:52 AM, Pierre Villard <pi...@gmail.com>> wrote:
By default, PutFile will set the ownership of the file to the user running the NiFi instance (nifi if NiFi is running as nifi user). Then, if you configured a different ownership in the processor configuration it'll try to set the ownership using the username you configured in the processor. What did you set in the processor parameters for owner/group and permission?
2018-05-30 14:39 GMT+02:00 James McMahon <js...@gmail.com>>:
Yes sir, sure does. In this instance my user nifi does indeed resolve at the OS level - I think that gives us some confidence it does resolve. The lookupPrincipalByName(owner) within the PutFile is where I believe the failure is rooted, but I do not understand how that function executes its lookup. I'm trying to dig deeper into that now.
On Wed, May 30, 2018 at 8:30 AM, Pierre Villard <pi...@gmail.com>> wrote:
I think we're saying the same :) Let me rephrase it differently: to set the owner of a file, the user needs to be resolved at OS level. If the user does not exist (from the OS point of view), NiFi won't be able to set the owner (even though the username is in the LDAP configured for NiFi authentication). Does it make more sense?
$ touch test.file
$ id toto
id: toto: no such user
$ chown toto test.file
chown: toto: illegal user name
Pierre
2018-05-30 14:13 GMT+02:00 James McMahon <js...@gmail.com>>:
I don't understand this:
"Until you can't resolve the user with OS commands, I don't think NiFi will be able to set the expected owner on the file"
Did you intend to say can there - don't we want to be able to resolve the user at the OS as an initial validation that we can get to the ldap and as prerequisite of Nifi being able to set the owner? And in this case I can indeed resolve at the OS level. That seems like a good thing to me.
On Wed, May 30, 2018 at 7:53 AM, Pierre Villard <pi...@gmail.com>> wrote:
It depends how your OS is configured, you could leverage tools like SSSD to resolve users against your LDAP but that's something to be configured at OS level.
Until you can't resolve the user with OS commands, I don't think NiFi will be able to set the expected owner on the file.
2018-05-30 11:54 GMT+02:00 James McMahon <js...@gmail.com>>:
Hello Pierre, and thank you. The user in this case - nifi - is not in the local /etc/passwd and is in the ldap. I presume this will force the id <username> to resolve using the ldap, if it does resolve? At the OS the id command returns the uid, the gid, and the groups to which user nifi has membership within the ldap.
On Wed, May 30, 2018 at 4:37 AM, Pierre Villard <pi...@gmail.com>> wrote:
Hi Jim,
LDAP for authentication and authorizations in NiFi has nothing to do with the processors.
How processors are running/working is completely independent to the authN/authZ model you configure for NiFi.
Regarding your error, I'd say that you get this error because user/group you're setting in the processor configuration cannot be resolved at OS level (even though they exist in the LDAP, but again, that's totally unrelated). Something you can quickly check: can you resolve the username/group on the host where you're using PutFile processor? What do you get if you execute the following command: id <username>?
Pierre
2018-05-30 1:14 GMT+02:00 Joe Witt <jo...@gmail.com>>:
jim
please only post to one list.
users is good for this.
thanks
joe
On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com>> wrote:
Good evening. I have recently migrated my nifi service host server from local resolution of users and groups to use an LDAP server. I configured login-identity-providers.xml and nifi.security.user.login.identity.provider. I verified my configuration is known to NiFi by first restarting my nifi service and then attempting a login to the URL by a user without a cert, forcing it to resolve using LDAP. This appeared to work.
I then attempted to set my file owner and file group in a PutFile to a user and a group that are each in the LDAP. The PutFile throws a Warning for both owner and group: java.nio.file.attribute.UserPrincipalNotFoundException. The file is still output by the processor. It appears to default the user and owner to nifi.
A cursory review of the PutFile source shows that PutFile employs getUserPrincipalLookupService() when it seemingly tries to validate the user and group.
How can I get this to resolve through the LDAP for the PutFile?
Thanks for any insights. -Jim
Re: User, Group in LDAP appear to be unknown to PutFile
Posted by James McMahon <js...@gmail.com>.
I did indeed configure PutFile as follows:
Permissions ..... 775
Owner ..... nifi
Group ..... ext_dev
When nifi is in local /etc/passwd and ext_dev is in local /etc/group, the
PutFile succeeds.
When neither exists in the local files, I get the Warning in both case and
the file is output with nifi:nifi (the default, as you mentioned Pierre).
When one is put back in the local file and the other left in the ldap, the
one in the local succeeds and the one only in the ldap still throws the
Warning and defaults on the output file.
We have also tried to reorder in our nsswitch.conf. For instance, changing
group: files ldap
to
group: ldap files
This did not change the results.
From the NiFi processor, user and group resolution do not appear to be
looking beyond a check in the local files.
I did double check that user nifi exists only in either the local file or
the ldap. Likewise for group ext_dev.
On Wed, May 30, 2018 at 8:52 AM, Pierre Villard <pierre.villard.fr@gmail.com
> wrote:
> By default, PutFile will set the ownership of the file to the user running
> the NiFi instance (nifi if NiFi is running as nifi user). Then, if you
> configured a different ownership in the processor configuration it'll try
> to set the ownership using the username you configured in the processor.
> What did you set in the processor parameters for owner/group and permission?
>
> 2018-05-30 14:39 GMT+02:00 James McMahon <js...@gmail.com>:
>
>> Yes sir, sure does. In this instance my user nifi does indeed resolve at
>> the OS level - I think that gives us some confidence it does resolve. The
>> lookupPrincipalByName(owner) within the PutFile is where I believe the
>> failure is rooted, but I do not understand how that function executes its
>> lookup. I'm trying to dig deeper into that now.
>>
>> On Wed, May 30, 2018 at 8:30 AM, Pierre Villard <
>> pierre.villard.fr@gmail.com> wrote:
>>
>>> I think we're saying the same :) Let me rephrase it differently: to set
>>> the owner of a file, the user needs to be resolved at OS level. If the user
>>> does not exist (from the OS point of view), NiFi won't be able to set the
>>> owner (even though the username is in the LDAP configured for NiFi
>>> authentication). Does it make more sense?
>>>
>>> $ touch test.file
>>> $ id toto
>>> id: toto: no such user
>>> $ chown toto test.file
>>> chown: toto: illegal user name
>>>
>>> Pierre
>>>
>>>
>>>
>>>
>>> 2018-05-30 14:13 GMT+02:00 James McMahon <js...@gmail.com>:
>>>
>>>> I don't understand this:
>>>> "Until you *can't* resolve the user with OS commands, I don't think
>>>> NiFi will be able to set the expected owner on the file"
>>>> Did you intend to say can there - don't we want to be able to resolve
>>>> the user at the OS as an initial validation that we can get to the ldap and
>>>> as prerequisite of Nifi being able to set the owner? And in this case I can
>>>> indeed resolve at the OS level. That seems like a good thing to me.
>>>>
>>>> On Wed, May 30, 2018 at 7:53 AM, Pierre Villard <
>>>> pierre.villard.fr@gmail.com> wrote:
>>>>
>>>>> It depends how your OS is configured, you could leverage tools like
>>>>> SSSD to resolve users against your LDAP but that's something to be
>>>>> configured at OS level.
>>>>> Until you can't resolve the user with OS commands, I don't think NiFi
>>>>> will be able to set the expected owner on the file.
>>>>>
>>>>> 2018-05-30 11:54 GMT+02:00 James McMahon <js...@gmail.com>:
>>>>>
>>>>>> Hello Pierre, and thank you. The user in this case - nifi - is not in
>>>>>> the local /etc/passwd and is in the ldap. I presume this will force the id
>>>>>> <username> to resolve using the ldap, if it does resolve? At the OS the id
>>>>>> command returns the uid, the gid, and the groups to which user nifi has
>>>>>> membership within the ldap.
>>>>>>
>>>>>> On Wed, May 30, 2018 at 4:37 AM, Pierre Villard <
>>>>>> pierre.villard.fr@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Jim,
>>>>>>>
>>>>>>> LDAP for authentication and authorizations in NiFi has nothing to do
>>>>>>> with the processors.
>>>>>>> How processors are running/working is completely independent to the
>>>>>>> authN/authZ model you configure for NiFi.
>>>>>>>
>>>>>>> Regarding your error, I'd say that you get this error because
>>>>>>> user/group you're setting in the processor configuration cannot be resolved
>>>>>>> at OS level (even though they exist in the LDAP, but again, that's totally
>>>>>>> unrelated). Something you can quickly check: can you resolve the
>>>>>>> username/group on the host where you're using PutFile processor? What do
>>>>>>> you get if you execute the following command: id <username>?
>>>>>>>
>>>>>>> Pierre
>>>>>>>
>>>>>>> 2018-05-30 1:14 GMT+02:00 Joe Witt <jo...@gmail.com>:
>>>>>>>
>>>>>>>> jim
>>>>>>>>
>>>>>>>> please only post to one list.
>>>>>>>>
>>>>>>>> users is good for this.
>>>>>>>>
>>>>>>>> thanks
>>>>>>>> joe
>>>>>>>>
>>>>>>>> On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Good evening. I have recently migrated my nifi service host server
>>>>>>>>> from local resolution of users and groups to use an LDAP server. I
>>>>>>>>> configured login-identity-providers.xml and nifi.security.user.login.identity.provider.
>>>>>>>>> I verified my configuration is known to NiFi by first restarting my nifi
>>>>>>>>> service and then attempting a login to the URL by a user without a cert,
>>>>>>>>> forcing it to resolve using LDAP. This appeared to work.
>>>>>>>>>
>>>>>>>>> I then attempted to set my file owner and file group in a PutFile
>>>>>>>>> to a user and a group that are each in the LDAP. The PutFile throws a
>>>>>>>>> Warning for both owner and group: java.nio.file.attribute.UserPrincipalNotFoundException.
>>>>>>>>> The file is still output by the processor. It appears to default the user
>>>>>>>>> and owner to nifi.
>>>>>>>>>
>>>>>>>>> A cursory review of the PutFile source shows that PutFile employs
>>>>>>>>> getUserPrincipalLookupService() when it seemingly tries to
>>>>>>>>> validate the user and group.
>>>>>>>>>
>>>>>>>>> How can I get this to resolve through the LDAP for the PutFile?
>>>>>>>>>
>>>>>>>>> Thanks for any insights. -Jim
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: User, Group in LDAP appear to be unknown to PutFile
Posted by Pierre Villard <pi...@gmail.com>.
By default, PutFile will set the ownership of the file to the user running
the NiFi instance (nifi if NiFi is running as nifi user). Then, if you
configured a different ownership in the processor configuration it'll try
to set the ownership using the username you configured in the processor.
What did you set in the processor parameters for owner/group and permission?
2018-05-30 14:39 GMT+02:00 James McMahon <js...@gmail.com>:
> Yes sir, sure does. In this instance my user nifi does indeed resolve at
> the OS level - I think that gives us some confidence it does resolve. The
> lookupPrincipalByName(owner) within the PutFile is where I believe the
> failure is rooted, but I do not understand how that function executes its
> lookup. I'm trying to dig deeper into that now.
>
> On Wed, May 30, 2018 at 8:30 AM, Pierre Villard <
> pierre.villard.fr@gmail.com> wrote:
>
>> I think we're saying the same :) Let me rephrase it differently: to set
>> the owner of a file, the user needs to be resolved at OS level. If the user
>> does not exist (from the OS point of view), NiFi won't be able to set the
>> owner (even though the username is in the LDAP configured for NiFi
>> authentication). Does it make more sense?
>>
>> $ touch test.file
>> $ id toto
>> id: toto: no such user
>> $ chown toto test.file
>> chown: toto: illegal user name
>>
>> Pierre
>>
>>
>>
>>
>> 2018-05-30 14:13 GMT+02:00 James McMahon <js...@gmail.com>:
>>
>>> I don't understand this:
>>> "Until you *can't* resolve the user with OS commands, I don't think
>>> NiFi will be able to set the expected owner on the file"
>>> Did you intend to say can there - don't we want to be able to resolve
>>> the user at the OS as an initial validation that we can get to the ldap and
>>> as prerequisite of Nifi being able to set the owner? And in this case I can
>>> indeed resolve at the OS level. That seems like a good thing to me.
>>>
>>> On Wed, May 30, 2018 at 7:53 AM, Pierre Villard <
>>> pierre.villard.fr@gmail.com> wrote:
>>>
>>>> It depends how your OS is configured, you could leverage tools like
>>>> SSSD to resolve users against your LDAP but that's something to be
>>>> configured at OS level.
>>>> Until you can't resolve the user with OS commands, I don't think NiFi
>>>> will be able to set the expected owner on the file.
>>>>
>>>> 2018-05-30 11:54 GMT+02:00 James McMahon <js...@gmail.com>:
>>>>
>>>>> Hello Pierre, and thank you. The user in this case - nifi - is not in
>>>>> the local /etc/passwd and is in the ldap. I presume this will force the id
>>>>> <username> to resolve using the ldap, if it does resolve? At the OS the id
>>>>> command returns the uid, the gid, and the groups to which user nifi has
>>>>> membership within the ldap.
>>>>>
>>>>> On Wed, May 30, 2018 at 4:37 AM, Pierre Villard <
>>>>> pierre.villard.fr@gmail.com> wrote:
>>>>>
>>>>>> Hi Jim,
>>>>>>
>>>>>> LDAP for authentication and authorizations in NiFi has nothing to do
>>>>>> with the processors.
>>>>>> How processors are running/working is completely independent to the
>>>>>> authN/authZ model you configure for NiFi.
>>>>>>
>>>>>> Regarding your error, I'd say that you get this error because
>>>>>> user/group you're setting in the processor configuration cannot be resolved
>>>>>> at OS level (even though they exist in the LDAP, but again, that's totally
>>>>>> unrelated). Something you can quickly check: can you resolve the
>>>>>> username/group on the host where you're using PutFile processor? What do
>>>>>> you get if you execute the following command: id <username>?
>>>>>>
>>>>>> Pierre
>>>>>>
>>>>>> 2018-05-30 1:14 GMT+02:00 Joe Witt <jo...@gmail.com>:
>>>>>>
>>>>>>> jim
>>>>>>>
>>>>>>> please only post to one list.
>>>>>>>
>>>>>>> users is good for this.
>>>>>>>
>>>>>>> thanks
>>>>>>> joe
>>>>>>>
>>>>>>> On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Good evening. I have recently migrated my nifi service host server
>>>>>>>> from local resolution of users and groups to use an LDAP server. I
>>>>>>>> configured login-identity-providers.xml and nifi.security.user.login.identity.provider.
>>>>>>>> I verified my configuration is known to NiFi by first restarting my nifi
>>>>>>>> service and then attempting a login to the URL by a user without a cert,
>>>>>>>> forcing it to resolve using LDAP. This appeared to work.
>>>>>>>>
>>>>>>>> I then attempted to set my file owner and file group in a PutFile
>>>>>>>> to a user and a group that are each in the LDAP. The PutFile throws a
>>>>>>>> Warning for both owner and group: java.nio.file.attribute.UserPrincipalNotFoundException.
>>>>>>>> The file is still output by the processor. It appears to default the user
>>>>>>>> and owner to nifi.
>>>>>>>>
>>>>>>>> A cursory review of the PutFile source shows that PutFile employs
>>>>>>>> getUserPrincipalLookupService() when it seemingly tries to
>>>>>>>> validate the user and group.
>>>>>>>>
>>>>>>>> How can I get this to resolve through the LDAP for the PutFile?
>>>>>>>>
>>>>>>>> Thanks for any insights. -Jim
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: User, Group in LDAP appear to be unknown to PutFile
Posted by Mike Thomsen <mi...@gmail.com>.
Shot in the dark, if you have a user named nifi in the LDAP and one in the
OS it might not actually be treated as the same unless the OS is using LDAP
to provide the user listing. Something as simple as /etc/users having a
password for "nifi" and the LDAP not having it or it being a different hash
could be throwing it off. Might want to look for something like that.
On Wed, May 30, 2018 at 8:40 AM James McMahon <js...@gmail.com> wrote:
> Yes sir, sure does. In this instance my user nifi does indeed resolve at
> the OS level - I think that gives us some confidence it does resolve. The
> lookupPrincipalByName(owner) within the PutFile is where I believe the
> failure is rooted, but I do not understand how that function executes its
> lookup. I'm trying to dig deeper into that now.
>
> On Wed, May 30, 2018 at 8:30 AM, Pierre Villard <
> pierre.villard.fr@gmail.com> wrote:
>
>> I think we're saying the same :) Let me rephrase it differently: to set
>> the owner of a file, the user needs to be resolved at OS level. If the user
>> does not exist (from the OS point of view), NiFi won't be able to set the
>> owner (even though the username is in the LDAP configured for NiFi
>> authentication). Does it make more sense?
>>
>> $ touch test.file
>> $ id toto
>> id: toto: no such user
>> $ chown toto test.file
>> chown: toto: illegal user name
>>
>> Pierre
>>
>>
>>
>>
>> 2018-05-30 14:13 GMT+02:00 James McMahon <js...@gmail.com>:
>>
>>> I don't understand this:
>>> "Until you *can't* resolve the user with OS commands, I don't think
>>> NiFi will be able to set the expected owner on the file"
>>> Did you intend to say can there - don't we want to be able to resolve
>>> the user at the OS as an initial validation that we can get to the ldap and
>>> as prerequisite of Nifi being able to set the owner? And in this case I can
>>> indeed resolve at the OS level. That seems like a good thing to me.
>>>
>>> On Wed, May 30, 2018 at 7:53 AM, Pierre Villard <
>>> pierre.villard.fr@gmail.com> wrote:
>>>
>>>> It depends how your OS is configured, you could leverage tools like
>>>> SSSD to resolve users against your LDAP but that's something to be
>>>> configured at OS level.
>>>> Until you can't resolve the user with OS commands, I don't think NiFi
>>>> will be able to set the expected owner on the file.
>>>>
>>>> 2018-05-30 11:54 GMT+02:00 James McMahon <js...@gmail.com>:
>>>>
>>>>> Hello Pierre, and thank you. The user in this case - nifi - is not in
>>>>> the local /etc/passwd and is in the ldap. I presume this will force the id
>>>>> <username> to resolve using the ldap, if it does resolve? At the OS the id
>>>>> command returns the uid, the gid, and the groups to which user nifi has
>>>>> membership within the ldap.
>>>>>
>>>>> On Wed, May 30, 2018 at 4:37 AM, Pierre Villard <
>>>>> pierre.villard.fr@gmail.com> wrote:
>>>>>
>>>>>> Hi Jim,
>>>>>>
>>>>>> LDAP for authentication and authorizations in NiFi has nothing to do
>>>>>> with the processors.
>>>>>> How processors are running/working is completely independent to the
>>>>>> authN/authZ model you configure for NiFi.
>>>>>>
>>>>>> Regarding your error, I'd say that you get this error because
>>>>>> user/group you're setting in the processor configuration cannot be resolved
>>>>>> at OS level (even though they exist in the LDAP, but again, that's totally
>>>>>> unrelated). Something you can quickly check: can you resolve the
>>>>>> username/group on the host where you're using PutFile processor? What do
>>>>>> you get if you execute the following command: id <username>?
>>>>>>
>>>>>> Pierre
>>>>>>
>>>>>> 2018-05-30 1:14 GMT+02:00 Joe Witt <jo...@gmail.com>:
>>>>>>
>>>>>>> jim
>>>>>>>
>>>>>>> please only post to one list.
>>>>>>>
>>>>>>> users is good for this.
>>>>>>>
>>>>>>> thanks
>>>>>>> joe
>>>>>>>
>>>>>>> On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Good evening. I have recently migrated my nifi service host server
>>>>>>>> from local resolution of users and groups to use an LDAP server. I
>>>>>>>> configured login-identity-providers.xml and
>>>>>>>> nifi.security.user.login.identity.provider. I verified my configuration is
>>>>>>>> known to NiFi by first restarting my nifi service and then attempting a
>>>>>>>> login to the URL by a user without a cert, forcing it to resolve using
>>>>>>>> LDAP. This appeared to work.
>>>>>>>>
>>>>>>>> I then attempted to set my file owner and file group in a PutFile
>>>>>>>> to a user and a group that are each in the LDAP. The PutFile throws a
>>>>>>>> Warning for both owner and group:
>>>>>>>> java.nio.file.attribute.UserPrincipalNotFoundException. The file is still
>>>>>>>> output by the processor. It appears to default the user and owner to nifi.
>>>>>>>>
>>>>>>>> A cursory review of the PutFile source shows that PutFile employs
>>>>>>>> getUserPrincipalLookupService() when it seemingly tries to validate the
>>>>>>>> user and group.
>>>>>>>>
>>>>>>>> How can I get this to resolve through the LDAP for the PutFile?
>>>>>>>>
>>>>>>>> Thanks for any insights. -Jim
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: User, Group in LDAP appear to be unknown to PutFile
Posted by James McMahon <js...@gmail.com>.
Yes sir, sure does. In this instance my user nifi does indeed resolve at
the OS level - I think that gives us some confidence it does resolve. The
lookupPrincipalByName(owner) within the PutFile is where I believe the
failure is rooted, but I do not understand how that function executes its
lookup. I'm trying to dig deeper into that now.
On Wed, May 30, 2018 at 8:30 AM, Pierre Villard <pierre.villard.fr@gmail.com
> wrote:
> I think we're saying the same :) Let me rephrase it differently: to set
> the owner of a file, the user needs to be resolved at OS level. If the user
> does not exist (from the OS point of view), NiFi won't be able to set the
> owner (even though the username is in the LDAP configured for NiFi
> authentication). Does it make more sense?
>
> $ touch test.file
> $ id toto
> id: toto: no such user
> $ chown toto test.file
> chown: toto: illegal user name
>
> Pierre
>
>
>
>
> 2018-05-30 14:13 GMT+02:00 James McMahon <js...@gmail.com>:
>
>> I don't understand this:
>> "Until you *can't* resolve the user with OS commands, I don't think NiFi
>> will be able to set the expected owner on the file"
>> Did you intend to say can there - don't we want to be able to resolve the
>> user at the OS as an initial validation that we can get to the ldap and as
>> prerequisite of Nifi being able to set the owner? And in this case I can
>> indeed resolve at the OS level. That seems like a good thing to me.
>>
>> On Wed, May 30, 2018 at 7:53 AM, Pierre Villard <
>> pierre.villard.fr@gmail.com> wrote:
>>
>>> It depends how your OS is configured, you could leverage tools like SSSD
>>> to resolve users against your LDAP but that's something to be configured at
>>> OS level.
>>> Until you can't resolve the user with OS commands, I don't think NiFi
>>> will be able to set the expected owner on the file.
>>>
>>> 2018-05-30 11:54 GMT+02:00 James McMahon <js...@gmail.com>:
>>>
>>>> Hello Pierre, and thank you. The user in this case - nifi - is not in
>>>> the local /etc/passwd and is in the ldap. I presume this will force the id
>>>> <username> to resolve using the ldap, if it does resolve? At the OS the id
>>>> command returns the uid, the gid, and the groups to which user nifi has
>>>> membership within the ldap.
>>>>
>>>> On Wed, May 30, 2018 at 4:37 AM, Pierre Villard <
>>>> pierre.villard.fr@gmail.com> wrote:
>>>>
>>>>> Hi Jim,
>>>>>
>>>>> LDAP for authentication and authorizations in NiFi has nothing to do
>>>>> with the processors.
>>>>> How processors are running/working is completely independent to the
>>>>> authN/authZ model you configure for NiFi.
>>>>>
>>>>> Regarding your error, I'd say that you get this error because
>>>>> user/group you're setting in the processor configuration cannot be resolved
>>>>> at OS level (even though they exist in the LDAP, but again, that's totally
>>>>> unrelated). Something you can quickly check: can you resolve the
>>>>> username/group on the host where you're using PutFile processor? What do
>>>>> you get if you execute the following command: id <username>?
>>>>>
>>>>> Pierre
>>>>>
>>>>> 2018-05-30 1:14 GMT+02:00 Joe Witt <jo...@gmail.com>:
>>>>>
>>>>>> jim
>>>>>>
>>>>>> please only post to one list.
>>>>>>
>>>>>> users is good for this.
>>>>>>
>>>>>> thanks
>>>>>> joe
>>>>>>
>>>>>> On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Good evening. I have recently migrated my nifi service host server
>>>>>>> from local resolution of users and groups to use an LDAP server. I
>>>>>>> configured login-identity-providers.xml and nifi.security.user.login.identity.provider.
>>>>>>> I verified my configuration is known to NiFi by first restarting my nifi
>>>>>>> service and then attempting a login to the URL by a user without a cert,
>>>>>>> forcing it to resolve using LDAP. This appeared to work.
>>>>>>>
>>>>>>> I then attempted to set my file owner and file group in a PutFile to
>>>>>>> a user and a group that are each in the LDAP. The PutFile throws a Warning
>>>>>>> for both owner and group: java.nio.file.attribute.UserPrincipalNotFoundException.
>>>>>>> The file is still output by the processor. It appears to default the user
>>>>>>> and owner to nifi.
>>>>>>>
>>>>>>> A cursory review of the PutFile source shows that PutFile employs
>>>>>>> getUserPrincipalLookupService() when it seemingly tries to validate
>>>>>>> the user and group.
>>>>>>>
>>>>>>> How can I get this to resolve through the LDAP for the PutFile?
>>>>>>>
>>>>>>> Thanks for any insights. -Jim
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: User, Group in LDAP appear to be unknown to PutFile
Posted by Pierre Villard <pi...@gmail.com>.
I think we're saying the same :) Let me rephrase it differently: to set the
owner of a file, the user needs to be resolved at OS level. If the user
does not exist (from the OS point of view), NiFi won't be able to set the
owner (even though the username is in the LDAP configured for NiFi
authentication). Does it make more sense?
$ touch test.file
$ id toto
id: toto: no such user
$ chown toto test.file
chown: toto: illegal user name
Pierre
2018-05-30 14:13 GMT+02:00 James McMahon <js...@gmail.com>:
> I don't understand this:
> "Until you *can't* resolve the user with OS commands, I don't think NiFi
> will be able to set the expected owner on the file"
> Did you intend to say can there - don't we want to be able to resolve the
> user at the OS as an initial validation that we can get to the ldap and as
> prerequisite of Nifi being able to set the owner? And in this case I can
> indeed resolve at the OS level. That seems like a good thing to me.
>
> On Wed, May 30, 2018 at 7:53 AM, Pierre Villard <
> pierre.villard.fr@gmail.com> wrote:
>
>> It depends how your OS is configured, you could leverage tools like SSSD
>> to resolve users against your LDAP but that's something to be configured at
>> OS level.
>> Until you can't resolve the user with OS commands, I don't think NiFi
>> will be able to set the expected owner on the file.
>>
>> 2018-05-30 11:54 GMT+02:00 James McMahon <js...@gmail.com>:
>>
>>> Hello Pierre, and thank you. The user in this case - nifi - is not in
>>> the local /etc/passwd and is in the ldap. I presume this will force the id
>>> <username> to resolve using the ldap, if it does resolve? At the OS the id
>>> command returns the uid, the gid, and the groups to which user nifi has
>>> membership within the ldap.
>>>
>>> On Wed, May 30, 2018 at 4:37 AM, Pierre Villard <
>>> pierre.villard.fr@gmail.com> wrote:
>>>
>>>> Hi Jim,
>>>>
>>>> LDAP for authentication and authorizations in NiFi has nothing to do
>>>> with the processors.
>>>> How processors are running/working is completely independent to the
>>>> authN/authZ model you configure for NiFi.
>>>>
>>>> Regarding your error, I'd say that you get this error because
>>>> user/group you're setting in the processor configuration cannot be resolved
>>>> at OS level (even though they exist in the LDAP, but again, that's totally
>>>> unrelated). Something you can quickly check: can you resolve the
>>>> username/group on the host where you're using PutFile processor? What do
>>>> you get if you execute the following command: id <username>?
>>>>
>>>> Pierre
>>>>
>>>> 2018-05-30 1:14 GMT+02:00 Joe Witt <jo...@gmail.com>:
>>>>
>>>>> jim
>>>>>
>>>>> please only post to one list.
>>>>>
>>>>> users is good for this.
>>>>>
>>>>> thanks
>>>>> joe
>>>>>
>>>>> On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Good evening. I have recently migrated my nifi service host server
>>>>>> from local resolution of users and groups to use an LDAP server. I
>>>>>> configured login-identity-providers.xml and nifi.security.user.login.identity.provider.
>>>>>> I verified my configuration is known to NiFi by first restarting my nifi
>>>>>> service and then attempting a login to the URL by a user without a cert,
>>>>>> forcing it to resolve using LDAP. This appeared to work.
>>>>>>
>>>>>> I then attempted to set my file owner and file group in a PutFile to
>>>>>> a user and a group that are each in the LDAP. The PutFile throws a Warning
>>>>>> for both owner and group: java.nio.file.attribute.UserPrincipalNotFoundException.
>>>>>> The file is still output by the processor. It appears to default the user
>>>>>> and owner to nifi.
>>>>>>
>>>>>> A cursory review of the PutFile source shows that PutFile employs
>>>>>> getUserPrincipalLookupService() when it seemingly tries to validate
>>>>>> the user and group.
>>>>>>
>>>>>> How can I get this to resolve through the LDAP for the PutFile?
>>>>>>
>>>>>> Thanks for any insights. -Jim
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: User, Group in LDAP appear to be unknown to PutFile
Posted by James McMahon <js...@gmail.com>.
I don't understand this:
"Until you *can't* resolve the user with OS commands, I don't think NiFi
will be able to set the expected owner on the file"
Did you intend to say can there - don't we want to be able to resolve the
user at the OS as an initial validation that we can get to the ldap and as
prerequisite of Nifi being able to set the owner? And in this case I can
indeed resolve at the OS level. That seems like a good thing to me.
On Wed, May 30, 2018 at 7:53 AM, Pierre Villard <pierre.villard.fr@gmail.com
> wrote:
> It depends how your OS is configured, you could leverage tools like SSSD
> to resolve users against your LDAP but that's something to be configured at
> OS level.
> Until you can't resolve the user with OS commands, I don't think NiFi will
> be able to set the expected owner on the file.
>
> 2018-05-30 11:54 GMT+02:00 James McMahon <js...@gmail.com>:
>
>> Hello Pierre, and thank you. The user in this case - nifi - is not in the
>> local /etc/passwd and is in the ldap. I presume this will force the id
>> <username> to resolve using the ldap, if it does resolve? At the OS the id
>> command returns the uid, the gid, and the groups to which user nifi has
>> membership within the ldap.
>>
>> On Wed, May 30, 2018 at 4:37 AM, Pierre Villard <
>> pierre.villard.fr@gmail.com> wrote:
>>
>>> Hi Jim,
>>>
>>> LDAP for authentication and authorizations in NiFi has nothing to do
>>> with the processors.
>>> How processors are running/working is completely independent to the
>>> authN/authZ model you configure for NiFi.
>>>
>>> Regarding your error, I'd say that you get this error because user/group
>>> you're setting in the processor configuration cannot be resolved at OS
>>> level (even though they exist in the LDAP, but again, that's totally
>>> unrelated). Something you can quickly check: can you resolve the
>>> username/group on the host where you're using PutFile processor? What do
>>> you get if you execute the following command: id <username>?
>>>
>>> Pierre
>>>
>>> 2018-05-30 1:14 GMT+02:00 Joe Witt <jo...@gmail.com>:
>>>
>>>> jim
>>>>
>>>> please only post to one list.
>>>>
>>>> users is good for this.
>>>>
>>>> thanks
>>>> joe
>>>>
>>>> On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com>
>>>> wrote:
>>>>
>>>>> Good evening. I have recently migrated my nifi service host server
>>>>> from local resolution of users and groups to use an LDAP server. I
>>>>> configured login-identity-providers.xml and nifi.security.user.login.identity.provider.
>>>>> I verified my configuration is known to NiFi by first restarting my nifi
>>>>> service and then attempting a login to the URL by a user without a cert,
>>>>> forcing it to resolve using LDAP. This appeared to work.
>>>>>
>>>>> I then attempted to set my file owner and file group in a PutFile to a
>>>>> user and a group that are each in the LDAP. The PutFile throws a Warning
>>>>> for both owner and group: java.nio.file.attribute.UserPrincipalNotFoundException.
>>>>> The file is still output by the processor. It appears to default the user
>>>>> and owner to nifi.
>>>>>
>>>>> A cursory review of the PutFile source shows that PutFile employs
>>>>> getUserPrincipalLookupService() when it seemingly tries to validate
>>>>> the user and group.
>>>>>
>>>>> How can I get this to resolve through the LDAP for the PutFile?
>>>>>
>>>>> Thanks for any insights. -Jim
>>>>>
>>>>
>>>
>>
>
Re: User, Group in LDAP appear to be unknown to PutFile
Posted by Pierre Villard <pi...@gmail.com>.
It depends how your OS is configured, you could leverage tools like SSSD to
resolve users against your LDAP but that's something to be configured at OS
level.
Until you can't resolve the user with OS commands, I don't think NiFi will
be able to set the expected owner on the file.
2018-05-30 11:54 GMT+02:00 James McMahon <js...@gmail.com>:
> Hello Pierre, and thank you. The user in this case - nifi - is not in the
> local /etc/passwd and is in the ldap. I presume this will force the id
> <username> to resolve using the ldap, if it does resolve? At the OS the id
> command returns the uid, the gid, and the groups to which user nifi has
> membership within the ldap.
>
> On Wed, May 30, 2018 at 4:37 AM, Pierre Villard <
> pierre.villard.fr@gmail.com> wrote:
>
>> Hi Jim,
>>
>> LDAP for authentication and authorizations in NiFi has nothing to do with
>> the processors.
>> How processors are running/working is completely independent to the
>> authN/authZ model you configure for NiFi.
>>
>> Regarding your error, I'd say that you get this error because user/group
>> you're setting in the processor configuration cannot be resolved at OS
>> level (even though they exist in the LDAP, but again, that's totally
>> unrelated). Something you can quickly check: can you resolve the
>> username/group on the host where you're using PutFile processor? What do
>> you get if you execute the following command: id <username>?
>>
>> Pierre
>>
>> 2018-05-30 1:14 GMT+02:00 Joe Witt <jo...@gmail.com>:
>>
>>> jim
>>>
>>> please only post to one list.
>>>
>>> users is good for this.
>>>
>>> thanks
>>> joe
>>>
>>> On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com>
>>> wrote:
>>>
>>>> Good evening. I have recently migrated my nifi service host server from
>>>> local resolution of users and groups to use an LDAP server. I configured
>>>> login-identity-providers.xml and nifi.security.user.login.identity.provider.
>>>> I verified my configuration is known to NiFi by first restarting my nifi
>>>> service and then attempting a login to the URL by a user without a cert,
>>>> forcing it to resolve using LDAP. This appeared to work.
>>>>
>>>> I then attempted to set my file owner and file group in a PutFile to a
>>>> user and a group that are each in the LDAP. The PutFile throws a Warning
>>>> for both owner and group: java.nio.file.attribute.UserPrincipalNotFoundException.
>>>> The file is still output by the processor. It appears to default the user
>>>> and owner to nifi.
>>>>
>>>> A cursory review of the PutFile source shows that PutFile employs
>>>> getUserPrincipalLookupService() when it seemingly tries to validate
>>>> the user and group.
>>>>
>>>> How can I get this to resolve through the LDAP for the PutFile?
>>>>
>>>> Thanks for any insights. -Jim
>>>>
>>>
>>
>
Re: User, Group in LDAP appear to be unknown to PutFile
Posted by James McMahon <js...@gmail.com>.
Hello Pierre, and thank you. The user in this case - nifi - is not in the
local /etc/passwd and is in the ldap. I presume this will force the id
<username> to resolve using the ldap, if it does resolve? At the OS the id
command returns the uid, the gid, and the groups to which user nifi has
membership within the ldap.
On Wed, May 30, 2018 at 4:37 AM, Pierre Villard <pierre.villard.fr@gmail.com
> wrote:
> Hi Jim,
>
> LDAP for authentication and authorizations in NiFi has nothing to do with
> the processors.
> How processors are running/working is completely independent to the
> authN/authZ model you configure for NiFi.
>
> Regarding your error, I'd say that you get this error because user/group
> you're setting in the processor configuration cannot be resolved at OS
> level (even though they exist in the LDAP, but again, that's totally
> unrelated). Something you can quickly check: can you resolve the
> username/group on the host where you're using PutFile processor? What do
> you get if you execute the following command: id <username>?
>
> Pierre
>
> 2018-05-30 1:14 GMT+02:00 Joe Witt <jo...@gmail.com>:
>
>> jim
>>
>> please only post to one list.
>>
>> users is good for this.
>>
>> thanks
>> joe
>>
>> On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com> wrote:
>>
>>> Good evening. I have recently migrated my nifi service host server from
>>> local resolution of users and groups to use an LDAP server. I configured
>>> login-identity-providers.xml and nifi.security.user.login.identity.provider.
>>> I verified my configuration is known to NiFi by first restarting my nifi
>>> service and then attempting a login to the URL by a user without a cert,
>>> forcing it to resolve using LDAP. This appeared to work.
>>>
>>> I then attempted to set my file owner and file group in a PutFile to a
>>> user and a group that are each in the LDAP. The PutFile throws a Warning
>>> for both owner and group: java.nio.file.attribute.UserPrincipalNotFoundException.
>>> The file is still output by the processor. It appears to default the user
>>> and owner to nifi.
>>>
>>> A cursory review of the PutFile source shows that PutFile employs
>>> getUserPrincipalLookupService() when it seemingly tries to validate the
>>> user and group.
>>>
>>> How can I get this to resolve through the LDAP for the PutFile?
>>>
>>> Thanks for any insights. -Jim
>>>
>>
>
Re: User, Group in LDAP appear to be unknown to PutFile
Posted by Pierre Villard <pi...@gmail.com>.
Hi Jim,
LDAP for authentication and authorizations in NiFi has nothing to do with
the processors.
How processors are running/working is completely independent to the
authN/authZ model you configure for NiFi.
Regarding your error, I'd say that you get this error because user/group
you're setting in the processor configuration cannot be resolved at OS
level (even though they exist in the LDAP, but again, that's totally
unrelated). Something you can quickly check: can you resolve the
username/group on the host where you're using PutFile processor? What do
you get if you execute the following command: id <username>?
Pierre
2018-05-30 1:14 GMT+02:00 Joe Witt <jo...@gmail.com>:
> jim
>
> please only post to one list.
>
> users is good for this.
>
> thanks
> joe
>
> On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com> wrote:
>
>> Good evening. I have recently migrated my nifi service host server from
>> local resolution of users and groups to use an LDAP server. I configured
>> login-identity-providers.xml and nifi.security.user.login.identity.provider.
>> I verified my configuration is known to NiFi by first restarting my nifi
>> service and then attempting a login to the URL by a user without a cert,
>> forcing it to resolve using LDAP. This appeared to work.
>>
>> I then attempted to set my file owner and file group in a PutFile to a
>> user and a group that are each in the LDAP. The PutFile throws a Warning
>> for both owner and group: java.nio.file.attribute.
>> UserPrincipalNotFoundException. The file is still output by the
>> processor. It appears to default the user and owner to nifi.
>>
>> A cursory review of the PutFile source shows that PutFile employs
>> getUserPrincipalLookupService() when it seemingly tries to validate the
>> user and group.
>>
>> How can I get this to resolve through the LDAP for the PutFile?
>>
>> Thanks for any insights. -Jim
>>
>
Re: User, Group in LDAP appear to be unknown to PutFile
Posted by Joe Witt <jo...@gmail.com>.
jim
please only post to one list.
users is good for this.
thanks
joe
On Tue, May 29, 2018, 3:54 PM James McMahon <js...@gmail.com> wrote:
> Good evening. I have recently migrated my nifi service host server from
> local resolution of users and groups to use an LDAP server. I configured
> login-identity-providers.xml and
> nifi.security.user.login.identity.provider. I verified my configuration is
> known to NiFi by first restarting my nifi service and then attempting a
> login to the URL by a user without a cert, forcing it to resolve using
> LDAP. This appeared to work.
>
> I then attempted to set my file owner and file group in a PutFile to a
> user and a group that are each in the LDAP. The PutFile throws a Warning
> for both owner and group:
> java.nio.file.attribute.UserPrincipalNotFoundException. The file is still
> output by the processor. It appears to default the user and owner to nifi.
>
> A cursory review of the PutFile source shows that PutFile employs
> getUserPrincipalLookupService() when it seemingly tries to validate the
> user and group.
>
> How can I get this to resolve through the LDAP for the PutFile?
>
> Thanks for any insights. -Jim
>