You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by gs...@tce.edu on 2010/02/18 06:57:51 UTC
Hadoop Security
hi all
i am doing my research in security issues of hadoop.i have noticed that
security can be attained under three heads like user authentication,data
transfer while running hadoop service and HDFS level.
Analyzed that kerberos cab be used for user authentication.when the user
wants to submit a job he/she can get delegation token followed by block
access token to access data from HDFS.So the client is overloaded with
initial 2 tickets (kerberos) TGT(Ticket grating Ticket),ST (service
ticket)followed by delegation token and block access token..Is that right??
suggestions plz...
regards
sujitha
-----------------------------------------
This email was sent using TCEMail Service.
Thiagarajar College of Engineering
Madurai-625 015, India
Re: Hadoop Security
Posted by Andrew Purtell <ap...@apache.org>.
See HBASE-1697 and go from there: https://issues.apache.org/jira/browse/HBASE-1697
We will try to track as closely to Hadoop ASF common as we can, same AAA top to bottom, HBase->RPC->HDFS.
- Andy
----- Original Message ----
> From: "Segel, Mike" <ms...@navteq.com>
> To: "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>
> Sent: Mon, February 22, 2010 7:18:47 AM
> Subject: RE: Hadoop Security
>
> Hi,
>
> Sorry for jumping in to this late, but has anyone thought about how this could
> be extended in to HBase?
> I realize this is Hadoop security, but eventually HBase and other apps that sit
> on top of hadoop will have to deal with security issues too.
>
> I'm not suggesting that a solution be worked out now, but that the solution for
> Hadoop can be extended to cover the apps that sit on top of Hadoop.
>
> Thx
>
> -Mike
>
> -----Original Message-----
> From: Owen O'Malley [mailto:omalley@apache.org]
> Sent: Sunday, February 21, 2010 4:02 PM
> To: common-dev@hadoop.apache.org
> Subject: Re: Hadoop Security
>
>
> On Feb 17, 2010, at 9:57 PM, gscse@tce.edu wrote:
>
> > Analyzed that kerberos cab be used for user authentication.when the
> > user
> > wants to submit a job he/she can get delegation token followed by
> > block
> > access token to access data from HDFS.So the client is overloaded with
> > initial 2 tickets (kerberos) TGT(Ticket grating Ticket),ST (service
> > ticket)followed by delegation token and block access token..Is that
> > right??
>
> When the user logs in to the system, they get a TGT. When they want to
> submit a job, they'll get two service tickets (one for the Name Node
> and one for the Job Tracker). They will get a delegation token from
> the NameNode and include that as part of the job. So in total,
> submitting a job should only take those 2 interactions with the
> Kerberos KDC.
>
> -- Owen
>
>
> The information contained in this communication may be CONFIDENTIAL and is
> intended only for the use of the recipient(s) named above. If you are not the
> intended recipient, you are hereby notified that any dissemination,
> distribution, or copying of this communication, or any of its contents, is
> strictly prohibited. If you have received this communication in error, please
> notify the sender and delete/destroy the original message and any copy of it
> from your computer or paper files.
Re: Hadoop Security
Posted by Owen O'Malley <om...@apache.org>.
On Feb 22, 2010, at 7:18 AM, Segel, Mike wrote:
> has anyone thought about how this could be extended in to HBase?
I don't think so, although you really should be asking on hbase-dev
instead of common-dev. *smile*
To the extent that HBase (and the other projects) use Common's rpc and
HttpServer, securing them will be easier, because the primitives will
be available. I believe HBase has their own clone of rpc and therefore
will need to rebase to Common's rpc if they want to secure the
connections.
In terms up upper level projects, we've been testing Pig with security
and only found a couple of issues. The vast majority of user code
won't need to change at all. I believe in both cases Pig was doing
something that they shouldn't have been doing. *smile*
> I realize this is Hadoop security, but eventually HBase and other
> apps that sit on top of hadoop will have to deal with security
> issues too.
Agreed.
> I'm not suggesting that a solution be worked out now, but that the
> solution for Hadoop can be extended to cover the apps that sit on
> top of Hadoop.
It is up to the individual sub-projects when they want to both
integrate with security. Yahoo is pushing really hard to get security
rolled out this year, so I'd expect the projects that they invest
heavily in (Common, HDFS, MapReduce, ZooKeeper, Pig) to move quickly.
-- Owen
Fw: Hadoop Security
Posted by Andrew Purtell <ap...@apache.org>.
>From common-dev@
----- Forwarded Message ----
> From: Andrew Purtell <ap...@apache.org>
> To: common-dev@hadoop.apache.org
> Sent: Tue, February 23, 2010 11:34:16 PM
> Subject: Re: Hadoop Security
>
> See HBASE-1697 and go from there:
> https://issues.apache.org/jira/browse/HBASE-1697
> We will try to track as closely to Hadoop ASF common as we can, same AAA top to
> bottom, HBase->RPC->HDFS.
>
> - Andy
>
>
>
> ----- Original Message ----
> > From: "Segel, Mike"
> > To: "common-dev@hadoop.apache.org"
> > Sent: Mon, February 22, 2010 7:18:47 AM
> > Subject: RE: Hadoop Security
> >
> > Hi,
> >
> > Sorry for jumping in to this late, but has anyone thought about how this could
>
> > be extended in to HBase?
> > I realize this is Hadoop security, but eventually HBase and other apps that
> sit
> > on top of hadoop will have to deal with security issues too.
> >
> > I'm not suggesting that a solution be worked out now, but that the solution
> for
> > Hadoop can be extended to cover the apps that sit on top of Hadoop.
> >
> > Thx
> >
> > -Mike
> >
> > -----Original Message-----
> > From: Owen O'Malley [mailto:omalley@apache.org]
> > Sent: Sunday, February 21, 2010 4:02 PM
> > To: common-dev@hadoop.apache.org
> > Subject: Re: Hadoop Security
> >
> >
> > On Feb 17, 2010, at 9:57 PM, gscse@tce.edu wrote:
> >
> > > Analyzed that kerberos cab be used for user authentication.when the
> > > user
> > > wants to submit a job he/she can get delegation token followed by
> > > block
> > > access token to access data from HDFS.So the client is overloaded with
> > > initial 2 tickets (kerberos) TGT(Ticket grating Ticket),ST (service
> > > ticket)followed by delegation token and block access token..Is that
> > > right??
> >
> > When the user logs in to the system, they get a TGT. When they want to
> > submit a job, they'll get two service tickets (one for the Name Node
> > and one for the Job Tracker). They will get a delegation token from
> > the NameNode and include that as part of the job. So in total,
> > submitting a job should only take those 2 interactions with the
> > Kerberos KDC.
> >
> > -- Owen
> >
> >
> > The information contained in this communication may be CONFIDENTIAL and is
> > intended only for the use of the recipient(s) named above. If you are not the
>
> > intended recipient, you are hereby notified that any dissemination,
> > distribution, or copying of this communication, or any of its contents, is
> > strictly prohibited. If you have received this communication in error, please
>
> > notify the sender and delete/destroy the original message and any copy of it
> > from your computer or paper files.
RE: Hadoop Security
Posted by "Segel, Mike" <ms...@navteq.com>.
Hi,
Sorry for jumping in to this late, but has anyone thought about how this could be extended in to HBase?
I realize this is Hadoop security, but eventually HBase and other apps that sit on top of hadoop will have to deal with security issues too.
I'm not suggesting that a solution be worked out now, but that the solution for Hadoop can be extended to cover the apps that sit on top of Hadoop.
Thx
-Mike
-----Original Message-----
From: Owen O'Malley [mailto:omalley@apache.org]
Sent: Sunday, February 21, 2010 4:02 PM
To: common-dev@hadoop.apache.org
Subject: Re: Hadoop Security
On Feb 17, 2010, at 9:57 PM, gscse@tce.edu wrote:
> Analyzed that kerberos cab be used for user authentication.when the
> user
> wants to submit a job he/she can get delegation token followed by
> block
> access token to access data from HDFS.So the client is overloaded with
> initial 2 tickets (kerberos) TGT(Ticket grating Ticket),ST (service
> ticket)followed by delegation token and block access token..Is that
> right??
When the user logs in to the system, they get a TGT. When they want to
submit a job, they'll get two service tickets (one for the Name Node
and one for the Job Tracker). They will get a delegation token from
the NameNode and include that as part of the job. So in total,
submitting a job should only take those 2 interactions with the
Kerberos KDC.
-- Owen
The information contained in this communication may be CONFIDENTIAL and is intended only for the use of the recipient(s) named above. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please notify the sender and delete/destroy the original message and any copy of it from your computer or paper files.
Re: Hadoop Security
Posted by Owen O'Malley <om...@apache.org>.
On Feb 17, 2010, at 9:57 PM, gscse@tce.edu wrote:
> Analyzed that kerberos cab be used for user authentication.when the
> user
> wants to submit a job he/she can get delegation token followed by
> block
> access token to access data from HDFS.So the client is overloaded with
> initial 2 tickets (kerberos) TGT(Ticket grating Ticket),ST (service
> ticket)followed by delegation token and block access token..Is that
> right??
When the user logs in to the system, they get a TGT. When they want to
submit a job, they'll get two service tickets (one for the Name Node
and one for the Job Tracker). They will get a delegation token from
the NameNode and include that as part of the job. So in total,
submitting a job should only take those 2 interactions with the
Kerberos KDC.
-- Owen