You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Di Li <os...@gmail.com> on 2018/04/04 18:15:32 UTC
Datanode does not need hdfs.headless.keytab ?
Hi folks,
I noticed hdfs.headless.keytab only exists on NameNode and HDFS client
node.
Could someone please share some details on why DN does not need the
hdfs.headless.keytab ? I thought we need it in order for DN to work against
NN.
Any negative impacts if I always include hdfs.headless.keytab on the DN
nodes (such as ensure HDFS client always cohost with DNs) ?
Thank you.
Di
Re: Datanode does not need hdfs.headless.keytab ?
Posted by Di Li <os...@gmail.com>.
Great ! Thanks Rob! I will try it out today and reach out if I hit issues.
Thank you for your help.
Di
On Wed, Apr 4, 2018 at 4:15 PM, Robert Levas <rl...@hortonworks.com> wrote:
> If you would like the HDFS keytab file installed on the same host as your
> component, you can add a reference to that Kerberos identity in your
> Kerberos.json file. Ideally this reference would be added to the
> "identities" section for the specific component. The declaration would
> look something like this:
>
> {
> "name": "custom_component_hdfs",
> "reference": "/HDFS/NAMENODE/hdfs"
> }
>
> For example:
>
> "components": [
> {
> "name": "MY_COMPONENT",
> ...
> "identities": [
> ...
> {
> "name": "custom_component_hdfs ",
> "reference": "/HDFS/NAMENODE/hdfs"
> }
> ...
> ],
> ...
> },
>
> I hope this helps.
>
> Rob
>
>
> On 4/4/18, 4:02 PM, "Di Li" <os...@gmail.com> wrote:
>
> Hi Rob,
>
> Thanks for the explanation. I don't have issues with DN per se. My case
> falls into the "*since then some services need to create directories
> and
> change permissions on them as the HDFS root user upon installation *
> category
> that you mentioned. I paired my service with DN assuming
> hdfs.headless.keytab would be available.
>
> Is that possible for my service's kerberos.json to define a dependency
> on
> hdfs.headless.keytab ? This way, I can still cohost my component with
> DN
> (hard requirement ...) but still have the keytab available (no need to
> modify HDFS kerberos.json)
>
> Thanks.
>
> Di
>
> On Wed, Apr 4, 2018 at 3:17 PM, Robert Levas <rl...@hortonworks.com>
> wrote:
>
> > The DN does not need to authenticate as the "root" HDFS user to
> perform
> > administrative tasks.
> >
> > A while back, we started an initiative to reduce the exposure of the
> HDFS
> > "root" user due to security concerns. In doing so, we tightened up
> where
> > we distribute the HDFS keytab file. However since then some services
> need
> > to create directories and change permissions on them as the HDFS
> root user
> > upon installation; and thus, the keytab file is being distributed
> more than
> > some security-conscious people would like. Until we find a way to
> > centralize the creation of these HDFS resources, we need to deal
> with this.
> >
> > You should not normally need the HDFS keytab file on DN hosts... are
> you
> > having an issue?
> >
> > Rob
> >
> >
> > On 4/4/18, 2:15 PM, "Di Li" <os...@gmail.com> wrote:
> >
> > Hi folks,
> >
> > I noticed hdfs.headless.keytab only exists on NameNode and HDFS
> client
> > node.
> >
> > Could someone please share some details on why DN does not need
> the
> > hdfs.headless.keytab ? I thought we need it in order for DN to
> work
> > against
> > NN.
> >
> > Any negative impacts if I always include hdfs.headless.keytab on
> the DN
> > nodes (such as ensure HDFS client always cohost with DNs) ?
> >
> > Thank you.
> >
> > Di
> >
> >
> >
>
>
>
Re: Datanode does not need hdfs.headless.keytab ?
Posted by Robert Levas <rl...@hortonworks.com>.
If you would like the HDFS keytab file installed on the same host as your component, you can add a reference to that Kerberos identity in your Kerberos.json file. Ideally this reference would be added to the "identities" section for the specific component. The declaration would look something like this:
{
"name": "custom_component_hdfs",
"reference": "/HDFS/NAMENODE/hdfs"
}
For example:
"components": [
{
"name": "MY_COMPONENT",
...
"identities": [
...
{
"name": "custom_component_hdfs ",
"reference": "/HDFS/NAMENODE/hdfs"
}
...
],
...
},
I hope this helps.
Rob
On 4/4/18, 4:02 PM, "Di Li" <os...@gmail.com> wrote:
Hi Rob,
Thanks for the explanation. I don't have issues with DN per se. My case
falls into the "*since then some services need to create directories and
change permissions on them as the HDFS root user upon installation * category
that you mentioned. I paired my service with DN assuming
hdfs.headless.keytab would be available.
Is that possible for my service's kerberos.json to define a dependency on
hdfs.headless.keytab ? This way, I can still cohost my component with DN
(hard requirement ...) but still have the keytab available (no need to
modify HDFS kerberos.json)
Thanks.
Di
On Wed, Apr 4, 2018 at 3:17 PM, Robert Levas <rl...@hortonworks.com> wrote:
> The DN does not need to authenticate as the "root" HDFS user to perform
> administrative tasks.
>
> A while back, we started an initiative to reduce the exposure of the HDFS
> "root" user due to security concerns. In doing so, we tightened up where
> we distribute the HDFS keytab file. However since then some services need
> to create directories and change permissions on them as the HDFS root user
> upon installation; and thus, the keytab file is being distributed more than
> some security-conscious people would like. Until we find a way to
> centralize the creation of these HDFS resources, we need to deal with this.
>
> You should not normally need the HDFS keytab file on DN hosts... are you
> having an issue?
>
> Rob
>
>
> On 4/4/18, 2:15 PM, "Di Li" <os...@gmail.com> wrote:
>
> Hi folks,
>
> I noticed hdfs.headless.keytab only exists on NameNode and HDFS client
> node.
>
> Could someone please share some details on why DN does not need the
> hdfs.headless.keytab ? I thought we need it in order for DN to work
> against
> NN.
>
> Any negative impacts if I always include hdfs.headless.keytab on the DN
> nodes (such as ensure HDFS client always cohost with DNs) ?
>
> Thank you.
>
> Di
>
>
>
Re: Datanode does not need hdfs.headless.keytab ?
Posted by Di Li <os...@gmail.com>.
Hi Rob,
Thanks for the explanation. I don't have issues with DN per se. My case
falls into the "*since then some services need to create directories and
change permissions on them as the HDFS root user upon installation * category
that you mentioned. I paired my service with DN assuming
hdfs.headless.keytab would be available.
Is that possible for my service's kerberos.json to define a dependency on
hdfs.headless.keytab ? This way, I can still cohost my component with DN
(hard requirement ...) but still have the keytab available (no need to
modify HDFS kerberos.json)
Thanks.
Di
On Wed, Apr 4, 2018 at 3:17 PM, Robert Levas <rl...@hortonworks.com> wrote:
> The DN does not need to authenticate as the "root" HDFS user to perform
> administrative tasks.
>
> A while back, we started an initiative to reduce the exposure of the HDFS
> "root" user due to security concerns. In doing so, we tightened up where
> we distribute the HDFS keytab file. However since then some services need
> to create directories and change permissions on them as the HDFS root user
> upon installation; and thus, the keytab file is being distributed more than
> some security-conscious people would like. Until we find a way to
> centralize the creation of these HDFS resources, we need to deal with this.
>
> You should not normally need the HDFS keytab file on DN hosts... are you
> having an issue?
>
> Rob
>
>
> On 4/4/18, 2:15 PM, "Di Li" <os...@gmail.com> wrote:
>
> Hi folks,
>
> I noticed hdfs.headless.keytab only exists on NameNode and HDFS client
> node.
>
> Could someone please share some details on why DN does not need the
> hdfs.headless.keytab ? I thought we need it in order for DN to work
> against
> NN.
>
> Any negative impacts if I always include hdfs.headless.keytab on the DN
> nodes (such as ensure HDFS client always cohost with DNs) ?
>
> Thank you.
>
> Di
>
>
>
Re: Datanode does not need hdfs.headless.keytab ?
Posted by Robert Levas <rl...@hortonworks.com>.
The DN does not need to authenticate as the "root" HDFS user to perform administrative tasks.
A while back, we started an initiative to reduce the exposure of the HDFS "root" user due to security concerns. In doing so, we tightened up where we distribute the HDFS keytab file. However since then some services need to create directories and change permissions on them as the HDFS root user upon installation; and thus, the keytab file is being distributed more than some security-conscious people would like. Until we find a way to centralize the creation of these HDFS resources, we need to deal with this.
You should not normally need the HDFS keytab file on DN hosts... are you having an issue?
Rob
On 4/4/18, 2:15 PM, "Di Li" <os...@gmail.com> wrote:
Hi folks,
I noticed hdfs.headless.keytab only exists on NameNode and HDFS client
node.
Could someone please share some details on why DN does not need the
hdfs.headless.keytab ? I thought we need it in order for DN to work against
NN.
Any negative impacts if I always include hdfs.headless.keytab on the DN
nodes (such as ensure HDFS client always cohost with DNs) ?
Thank you.
Di