You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Hafiz Mujadid <ha...@gmail.com> on 2016/05/30 16:04:17 UTC
KMS for hadoop
Hi,
I am new to hadoop and want to enable KMS for hadoop. I have read this
<http://aajisaka.github.io/hadoop-project/hadoop-kms/index.html> kms
documentation on hadoop but unable to get idea how to get started. Is there
any detailed getting started guide for kms? does KMS is supported by
default and we only need to enable it?
Thanks
RE: KMS for hadoop
Posted by "Dietrich, Paul" <pa...@honeywell.com>.
I’m not going to be able to help much because I used Kerberos and Apache Ranger.
In the end I found a security problem that I posted here with no resolution so I am not using KMS for now.
Paul
From: Aneela Saleem [mailto:aneela@platalytics.com]
Sent: Thursday, June 02, 2016 12:43 AM
To: Dietrich, Paul <pa...@honeywell.com>
Cc: Hafiz Mujadid <ha...@gmail.com>; user@hadoop.apache.org
Subject: Re: KMS for hadoop
Hi Paul,
Can you please guide me what are the basic steps to configure KMS with Hadoop. Because the documentation here<https://hadoop.apache.org/docs/stable/hadoop-kms/index.html> is very brief. And i have non-kerberized cluster. Can you please guide us to take a very well start.
Thanks
On Wed, Jun 1, 2016 at 4:14 PM, Dietrich, Paul <pa...@honeywell.com>> wrote:
It can be setup standalone. The configuration property hadoop.kms.authentication.type has a default value of simple.
Paul
From: Hafiz Mujadid [mailto:hafizmujadid00@gmail.com<ma...@gmail.com>]
Sent: Wednesday, June 01, 2016 1:49 AM
To: Dietrich, Paul <pa...@honeywell.com>>
Cc: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: KMS for hadoop
Thanks Paul for your response.
Do I need to setup Kerberos before enabling KMS? Or KMS can be setup standalone ?
Thanks
On Tue, May 31, 2016 at 6:56 PM, Dietrich, Paul <pa...@honeywell.com>> wrote:
Hafiz,
I didn’t find such a guide, but used documentation from Cloudera and Hortonworks to augment what you found. KMS is part of Hadoop (in later versions), so it is just a matter of setting the configuration parameters to enable it. One thing to note is that KMS should be part of a secure cluster so you’ll need to do the necessary steps to setup Kerberos et al, which could restrict your universe of tools that you use. Also using the file based keystore is probably not a good idea if you are looking to use this in an “enterprise” environment. Being able to secure and manage a key server is not trivial.
Paul
From: Hafiz Mujadid [mailto:hafizmujadid00@gmail.com<ma...@gmail.com>]
Sent: Monday, May 30, 2016 11:04 AM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: KMS for hadoop
Hi,
I am new to hadoop and want to enable KMS for hadoop. I have read this<http://aajisaka.github.io/hadoop-project/hadoop-kms/index.html> kms documentation on hadoop but unable to get idea how to get started. Is there any detailed getting started guide for kms? does KMS is supported by default and we only need to enable it?
Thanks
--
Regards: HAFIZ MUJADID
Re: KMS for hadoop
Posted by Aneela Saleem <an...@platalytics.com>.
Hi Paul,
Can you please guide me what are the basic steps to configure KMS with
Hadoop. Because the documentation here
<https://hadoop.apache.org/docs/stable/hadoop-kms/index.html> is very
brief. And i have non-kerberized cluster. Can you please guide us to take a
very well start.
Thanks
On Wed, Jun 1, 2016 at 4:14 PM, Dietrich, Paul <pa...@honeywell.com>
wrote:
> It can be setup standalone. The configuration property
> hadoop.kms.authentication.type has a default value of simple.
>
>
>
> Paul
>
>
>
> *From:* Hafiz Mujadid [mailto:hafizmujadid00@gmail.com]
> *Sent:* Wednesday, June 01, 2016 1:49 AM
> *To:* Dietrich, Paul <pa...@honeywell.com>
> *Cc:* user@hadoop.apache.org
> *Subject:* Re: KMS for hadoop
>
>
>
> Thanks Paul for your response.
>
> Do I need to setup Kerberos before enabling KMS? Or KMS can be setup
> standalone ?
>
> Thanks
>
>
>
> On Tue, May 31, 2016 at 6:56 PM, Dietrich, Paul <
> paul.dietrich@honeywell.com> wrote:
>
> Hafiz,
>
> I didn’t find such a guide, but used documentation from Cloudera and
> Hortonworks to augment what you found. KMS is part of Hadoop (in later
> versions), so it is just a matter of setting the configuration parameters
> to enable it. One thing to note is that KMS should be part of a secure
> cluster so you’ll need to do the necessary steps to setup Kerberos et al,
> which could restrict your universe of tools that you use. Also using the
> file based keystore is probably not a good idea if you are looking to use
> this in an “enterprise” environment. Being able to secure and manage a key
> server is not trivial.
>
>
>
> Paul
>
>
>
> *From:* Hafiz Mujadid [mailto:hafizmujadid00@gmail.com]
> *Sent:* Monday, May 30, 2016 11:04 AM
> *To:* user@hadoop.apache.org
> *Subject:* KMS for hadoop
>
>
>
> Hi,
>
> I am new to hadoop and want to enable KMS for hadoop. I have read this
> <http://aajisaka.github.io/hadoop-project/hadoop-kms/index.html> kms
> documentation on hadoop but unable to get idea how to get started. Is there
> any detailed getting started guide for kms? does KMS is supported by
> default and we only need to enable it?
>
>
>
>
>
> Thanks
>
>
>
>
>
> --
>
> Regards: HAFIZ MUJADID
>
RE: KMS for hadoop
Posted by "Dietrich, Paul" <pa...@honeywell.com>.
It can be setup standalone. The configuration property hadoop.kms.authentication.type has a default value of simple.
Paul
From: Hafiz Mujadid [mailto:hafizmujadid00@gmail.com]
Sent: Wednesday, June 01, 2016 1:49 AM
To: Dietrich, Paul <pa...@honeywell.com>
Cc: user@hadoop.apache.org
Subject: Re: KMS for hadoop
Thanks Paul for your response.
Do I need to setup Kerberos before enabling KMS? Or KMS can be setup standalone ?
Thanks
On Tue, May 31, 2016 at 6:56 PM, Dietrich, Paul <pa...@honeywell.com>> wrote:
Hafiz,
I didn’t find such a guide, but used documentation from Cloudera and Hortonworks to augment what you found. KMS is part of Hadoop (in later versions), so it is just a matter of setting the configuration parameters to enable it. One thing to note is that KMS should be part of a secure cluster so you’ll need to do the necessary steps to setup Kerberos et al, which could restrict your universe of tools that you use. Also using the file based keystore is probably not a good idea if you are looking to use this in an “enterprise” environment. Being able to secure and manage a key server is not trivial.
Paul
From: Hafiz Mujadid [mailto:hafizmujadid00@gmail.com<ma...@gmail.com>]
Sent: Monday, May 30, 2016 11:04 AM
To: user@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: KMS for hadoop
Hi,
I am new to hadoop and want to enable KMS for hadoop. I have read this<http://aajisaka.github.io/hadoop-project/hadoop-kms/index.html> kms documentation on hadoop but unable to get idea how to get started. Is there any detailed getting started guide for kms? does KMS is supported by default and we only need to enable it?
Thanks
--
Regards: HAFIZ MUJADID
Re: KMS for hadoop
Posted by Hafiz Mujadid <ha...@gmail.com>.
Thanks Paul for your response.
Do I need to setup Kerberos before enabling KMS? Or KMS can be setup
standalone ?
Thanks
On Tue, May 31, 2016 at 6:56 PM, Dietrich, Paul <paul.dietrich@honeywell.com
> wrote:
> Hafiz,
>
> I didn’t find such a guide, but used documentation from Cloudera and
> Hortonworks to augment what you found. KMS is part of Hadoop (in later
> versions), so it is just a matter of setting the configuration parameters
> to enable it. One thing to note is that KMS should be part of a secure
> cluster so you’ll need to do the necessary steps to setup Kerberos et al,
> which could restrict your universe of tools that you use. Also using the
> file based keystore is probably not a good idea if you are looking to use
> this in an “enterprise” environment. Being able to secure and manage a key
> server is not trivial.
>
>
>
> Paul
>
>
>
> *From:* Hafiz Mujadid [mailto:hafizmujadid00@gmail.com]
> *Sent:* Monday, May 30, 2016 11:04 AM
> *To:* user@hadoop.apache.org
> *Subject:* KMS for hadoop
>
>
>
> Hi,
>
> I am new to hadoop and want to enable KMS for hadoop. I have read this
> <http://aajisaka.github.io/hadoop-project/hadoop-kms/index.html> kms
> documentation on hadoop but unable to get idea how to get started. Is there
> any detailed getting started guide for kms? does KMS is supported by
> default and we only need to enable it?
>
>
>
>
>
> Thanks
>
--
Regards: HAFIZ MUJADID
RE: KMS for hadoop
Posted by "Dietrich, Paul" <pa...@honeywell.com>.
Hafiz,
I didn’t find such a guide, but used documentation from Cloudera and Hortonworks to augment what you found. KMS is part of Hadoop (in later versions), so it is just a matter of setting the configuration parameters to enable it. One thing to note is that KMS should be part of a secure cluster so you’ll need to do the necessary steps to setup Kerberos et al, which could restrict your universe of tools that you use. Also using the file based keystore is probably not a good idea if you are looking to use this in an “enterprise” environment. Being able to secure and manage a key server is not trivial.
Paul
From: Hafiz Mujadid [mailto:hafizmujadid00@gmail.com]
Sent: Monday, May 30, 2016 11:04 AM
To: user@hadoop.apache.org
Subject: KMS for hadoop
Hi,
I am new to hadoop and want to enable KMS for hadoop. I have read this<http://aajisaka.github.io/hadoop-project/hadoop-kms/index.html> kms documentation on hadoop but unable to get idea how to get started. Is there any detailed getting started guide for kms? does KMS is supported by default and we only need to enable it?
Thanks