New groups for HDFS user? And scope of that user?

["Alex.mclintock" <al...@gmail.com>]

I am running three kerberized Hadoop clusters and am moving from Ambari 1.7 to 2.0.1 
(Actually Hortonworks HDP 2.2.0 to 2.2.6 on RHEL6)

I have two main questions around how Ambari controls certain users -in particular hdfs and Ambari-qa

Firstly, how do I add the hdfs user to new groups? I want to use to a java debugging tool and want the process running datanodes to be able to write to certain directories. If I add the groups manually then I think Ambari overwrites them later on.

Secondly. The hdfs user is defined as headless, or site wide. Does this mean that it is supposed to be shared among multiple clusters on the same site? But I have one Ambari instance per cluster, and they all seem to think they own the Hdfs user. This probably does not matter to most people until you realise that when kerberized Ambari controls its keytab and thus its ability to authenticate with the KDC. ONe Ambari instance might invalidate the hdfs user in another

Alex


Sent from my iPad

Re: New groups for HDFS user? And scope of that user?

[Robert Levas <rl...@hortonworks.com>]

Hi AlexŠ

I don¹t think I can answer you question about groups. I assume it is more
than just setting the group for the local account.

Regarding the headless Kerberos identitiesŠ if Ambari is to manage the
Kerberos identities, and multiple clusters are setup using the same KDC,
then the headless users need to be unique across the different clusters.
In Ambari 2.1 (release TBD) Ambari, by default, will add the cluster name
to the relevant principal names to avoid any collisions - assuming your
clusters have unique names.  In Ambari 2.0.x, this is not done for you
automatically and you should explicitly add the cluster name (or some
other unique-ifer) to the headless principal names.

When upgrading to Ambari 2.x, you can choose to not tell Ambari about your
Kerberos configuration, and things will work fine, but Ambari will not
behave properly when you add new hosts or services, nor will it
automatically set Kerberos-specific configuration properties.   If you
want Ambari to do this for you, you should enable Kerberos via the Enable
Kerberos Wizard, where you will be walked through the steps necessary to
configure the cluster for Kerberos.  When you reach the Configure
Principals page you will need to find the headless principals and add the
unique-ifier to each of them. Most should be in the Ambari Principals
section, but some may be in the Advanced section (this issue is fixed in
Ambari 2.1).  If you simply want to add the cluster name, you can add
³-${cluster_name}² to the principal name portion of the relevant
principals. Then when Ambari (re)creates the Kerberos identities, the
principal names for the headless Kerberos identities will be unique, thus
avoiding collisions.

If the principal names do happen to collide, one cluster will work, but
there other(s) will fail.  This is because Ambari will change the password
for any existing identity so that it can properly create the keytab file
for it. 

I hope this helps,

Rob




On 6/19/15, 5:19 AM, "Alex.mclintock" <al...@gmail.com> wrote:

>I am running three kerberized Hadoop clusters and am moving from Ambari
>1.7 to 2.0.1 
>(Actually Hortonworks HDP 2.2.0 to 2.2.6 on RHEL6)
>
>I have two main questions around how Ambari controls certain users -in
>particular hdfs and Ambari-qa
>
>Firstly, how do I add the hdfs user to new groups? I want to use to a
>java debugging tool and want the process running datanodes to be able to
>write to certain directories. If I add the groups manually then I think
>Ambari overwrites them later on.
>
>Secondly. The hdfs user is defined as headless, or site wide. Does this
>mean that it is supposed to be shared among multiple clusters on the same
>site? But I have one Ambari instance per cluster, and they all seem to
>think they own the Hdfs user. This probably does not matter to most
>people until you realise that when kerberized Ambari controls its keytab
>and thus its ability to authenticate with the KDC. ONe Ambari instance
>might invalidate the hdfs user in another
>
>Alex
>
>
>Sent from my iPad