You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Manoj Samel <ma...@gmail.com> on 2014/01/07 23:55:41 UTC
Ways to manage user accounts on hadoop cluster when using kerberos security
Hi,
>From the documentation + code, "when kerberos is enabled, all tasks are
run as the end user (e..g as user "joe" and not as hadoop user "mapred")
using the task-controller (which is setuid root and when it runs, it does a
setuid/setgid etc. to Joe and his groups ). For this to work, user "joe"
linux account has to be present on all nodes of the cluster."
In a environment with large and dynamic user population; it is not
practical to add every end user to every node of the cluster (and drop user
when end user is deactivated etc.)
What are other options get this working ?
I am assuming that if the users are in a LDAP, can using the PAM for LDAP
solve the issue.
Any other suggestions?
--
Thanks,
Manoj
Re: Ways to manage user accounts on hadoop cluster when using kerberos security
Posted by Jay Vyas <ja...@gmail.com>.
I recently found a pretty simple and easy way to set ldap up for my machines on rhel and wrote it up using jumpbox and authconfig.
If you are in the cloud and only need a quick easy ldap idh and nssswitch setup, this is I think the easiest / cheapest way to do it.
I know rhel and fedora come with authconfig not sure about the other Linux distros:
http://jayunit100.blogspot.com/2013/12/an-easy-way-to-centralize.html?m=1
> On Jan 8, 2014, at 1:22 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
>
>
>> On Jan 7, 2014, at 2:55 PM, Manoj Samel <ma...@gmail.com> wrote:
>>
>> I am assuming that if the users are in a LDAP, can using the PAM for LDAP solve the issue.
>
>
> That's how I've seen this issue addressed.
>
> +Vinod
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Ways to manage user accounts on hadoop cluster when using kerberos security
Posted by Jay Vyas <ja...@gmail.com>.
I recently found a pretty simple and easy way to set ldap up for my machines on rhel and wrote it up using jumpbox and authconfig.
If you are in the cloud and only need a quick easy ldap idh and nssswitch setup, this is I think the easiest / cheapest way to do it.
I know rhel and fedora come with authconfig not sure about the other Linux distros:
http://jayunit100.blogspot.com/2013/12/an-easy-way-to-centralize.html?m=1
> On Jan 8, 2014, at 1:22 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
>
>
>> On Jan 7, 2014, at 2:55 PM, Manoj Samel <ma...@gmail.com> wrote:
>>
>> I am assuming that if the users are in a LDAP, can using the PAM for LDAP solve the issue.
>
>
> That's how I've seen this issue addressed.
>
> +Vinod
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Ways to manage user accounts on hadoop cluster when using kerberos security
Posted by Jay Vyas <ja...@gmail.com>.
I recently found a pretty simple and easy way to set ldap up for my machines on rhel and wrote it up using jumpbox and authconfig.
If you are in the cloud and only need a quick easy ldap idh and nssswitch setup, this is I think the easiest / cheapest way to do it.
I know rhel and fedora come with authconfig not sure about the other Linux distros:
http://jayunit100.blogspot.com/2013/12/an-easy-way-to-centralize.html?m=1
> On Jan 8, 2014, at 1:22 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
>
>
>> On Jan 7, 2014, at 2:55 PM, Manoj Samel <ma...@gmail.com> wrote:
>>
>> I am assuming that if the users are in a LDAP, can using the PAM for LDAP solve the issue.
>
>
> That's how I've seen this issue addressed.
>
> +Vinod
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Ways to manage user accounts on hadoop cluster when using kerberos security
Posted by Jay Vyas <ja...@gmail.com>.
I recently found a pretty simple and easy way to set ldap up for my machines on rhel and wrote it up using jumpbox and authconfig.
If you are in the cloud and only need a quick easy ldap idh and nssswitch setup, this is I think the easiest / cheapest way to do it.
I know rhel and fedora come with authconfig not sure about the other Linux distros:
http://jayunit100.blogspot.com/2013/12/an-easy-way-to-centralize.html?m=1
> On Jan 8, 2014, at 1:22 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
>
>
>> On Jan 7, 2014, at 2:55 PM, Manoj Samel <ma...@gmail.com> wrote:
>>
>> I am assuming that if the users are in a LDAP, can using the PAM for LDAP solve the issue.
>
>
> That's how I've seen this issue addressed.
>
> +Vinod
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Ways to manage user accounts on hadoop cluster when using kerberos security
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
On Jan 7, 2014, at 2:55 PM, Manoj Samel <ma...@gmail.com> wrote:
> I am assuming that if the users are in a LDAP, can using the PAM for LDAP solve the issue.
That's how I've seen this issue addressed.
+Vinod
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Ways to manage user accounts on hadoop cluster when using
kerberos security
Posted by bc Wong <bc...@cloudera.com>.
LDAP/AD is pretty much it. You can also have Kerberos authenticate directly
to AD, or set up one-way trust between AD and MIT Kerberos. There are other
identity management systems that basically implement the same. At the end
of the day, you need to have (1) users in KDC (2) users on the nodes, and
(3) user-group mapping. And it makes sense for all three to come from the
same system.
Cheers,
bc
On Tue, Jan 7, 2014 at 2:55 PM, Manoj Samel <ma...@gmail.com> wrote:
> Hi,
>
> From the documentation + code, "when kerberos is enabled, all tasks are
> run as the end user (e..g as user "joe" and not as hadoop user "mapred")
> using the task-controller (which is setuid root and when it runs, it does a
> setuid/setgid etc. to Joe and his groups ). For this to work, user "joe"
> linux account has to be present on all nodes of the cluster."
>
> In a environment with large and dynamic user population; it is not
> practical to add every end user to every node of the cluster (and drop user
> when end user is deactivated etc.)
>
> What are other options get this working ?
>
> I am assuming that if the users are in a LDAP, can using the PAM for LDAP
> solve the issue.
>
> Any other suggestions?
>
> --
> Thanks,
>
> Manoj
>
Re: Ways to manage user accounts on hadoop cluster when using kerberos security
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
On Jan 7, 2014, at 2:55 PM, Manoj Samel <ma...@gmail.com> wrote:
> I am assuming that if the users are in a LDAP, can using the PAM for LDAP solve the issue.
That's how I've seen this issue addressed.
+Vinod
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Ways to manage user accounts on hadoop cluster when using kerberos security
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
On Jan 7, 2014, at 2:55 PM, Manoj Samel <ma...@gmail.com> wrote:
> I am assuming that if the users are in a LDAP, can using the PAM for LDAP solve the issue.
That's how I've seen this issue addressed.
+Vinod
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Ways to manage user accounts on hadoop cluster when using kerberos security
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
On Jan 7, 2014, at 2:55 PM, Manoj Samel <ma...@gmail.com> wrote:
> I am assuming that if the users are in a LDAP, can using the PAM for LDAP solve the issue.
That's how I've seen this issue addressed.
+Vinod
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Ways to manage user accounts on hadoop cluster when using
kerberos security
Posted by bc Wong <bc...@cloudera.com>.
LDAP/AD is pretty much it. You can also have Kerberos authenticate directly
to AD, or set up one-way trust between AD and MIT Kerberos. There are other
identity management systems that basically implement the same. At the end
of the day, you need to have (1) users in KDC (2) users on the nodes, and
(3) user-group mapping. And it makes sense for all three to come from the
same system.
Cheers,
bc
On Tue, Jan 7, 2014 at 2:55 PM, Manoj Samel <ma...@gmail.com> wrote:
> Hi,
>
> From the documentation + code, "when kerberos is enabled, all tasks are
> run as the end user (e..g as user "joe" and not as hadoop user "mapred")
> using the task-controller (which is setuid root and when it runs, it does a
> setuid/setgid etc. to Joe and his groups ). For this to work, user "joe"
> linux account has to be present on all nodes of the cluster."
>
> In a environment with large and dynamic user population; it is not
> practical to add every end user to every node of the cluster (and drop user
> when end user is deactivated etc.)
>
> What are other options get this working ?
>
> I am assuming that if the users are in a LDAP, can using the PAM for LDAP
> solve the issue.
>
> Any other suggestions?
>
> --
> Thanks,
>
> Manoj
>
Re: Ways to manage user accounts on hadoop cluster when using
kerberos security
Posted by bc Wong <bc...@cloudera.com>.
LDAP/AD is pretty much it. You can also have Kerberos authenticate directly
to AD, or set up one-way trust between AD and MIT Kerberos. There are other
identity management systems that basically implement the same. At the end
of the day, you need to have (1) users in KDC (2) users on the nodes, and
(3) user-group mapping. And it makes sense for all three to come from the
same system.
Cheers,
bc
On Tue, Jan 7, 2014 at 2:55 PM, Manoj Samel <ma...@gmail.com> wrote:
> Hi,
>
> From the documentation + code, "when kerberos is enabled, all tasks are
> run as the end user (e..g as user "joe" and not as hadoop user "mapred")
> using the task-controller (which is setuid root and when it runs, it does a
> setuid/setgid etc. to Joe and his groups ). For this to work, user "joe"
> linux account has to be present on all nodes of the cluster."
>
> In a environment with large and dynamic user population; it is not
> practical to add every end user to every node of the cluster (and drop user
> when end user is deactivated etc.)
>
> What are other options get this working ?
>
> I am assuming that if the users are in a LDAP, can using the PAM for LDAP
> solve the issue.
>
> Any other suggestions?
>
> --
> Thanks,
>
> Manoj
>
Re: Ways to manage user accounts on hadoop cluster when using
kerberos security
Posted by bc Wong <bc...@cloudera.com>.
LDAP/AD is pretty much it. You can also have Kerberos authenticate directly
to AD, or set up one-way trust between AD and MIT Kerberos. There are other
identity management systems that basically implement the same. At the end
of the day, you need to have (1) users in KDC (2) users on the nodes, and
(3) user-group mapping. And it makes sense for all three to come from the
same system.
Cheers,
bc
On Tue, Jan 7, 2014 at 2:55 PM, Manoj Samel <ma...@gmail.com> wrote:
> Hi,
>
> From the documentation + code, "when kerberos is enabled, all tasks are
> run as the end user (e..g as user "joe" and not as hadoop user "mapred")
> using the task-controller (which is setuid root and when it runs, it does a
> setuid/setgid etc. to Joe and his groups ). For this to work, user "joe"
> linux account has to be present on all nodes of the cluster."
>
> In a environment with large and dynamic user population; it is not
> practical to add every end user to every node of the cluster (and drop user
> when end user is deactivated etc.)
>
> What are other options get this working ?
>
> I am assuming that if the users are in a LDAP, can using the PAM for LDAP
> solve the issue.
>
> Any other suggestions?
>
> --
> Thanks,
>
> Manoj
>