You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Dmitry Pushkarev <um...@stanford.edu> on 2008/10/05 08:54:12 UTC
Hadoop and security.
Dear hadoop users,
I'm lucky to work in academic environment where information security is not
the question. However, I'm sure that most of the hadoop users aren't.
Here is the question: how secure hadoop is? (or let's say foolproof)
Here is the answer: http://www.google.com/search?client=opera
<http://www.google.com/search?client=opera&rls=en&q=Hadoop+Map/Reduce+Admini
stration&sourceid=opera&ie=utf-8&oe=utf-8>
&rls=en&q=Hadoop+Map/Reduce+Administration&sourceid=opera&ie=utf-8&oe=utf-8
not quite.
What we're seeing here is open hadoop cluster, where anyone who capable of
installing hadoop and changing his username to webcrawl can use their
cluster and read their data, even though firewall is perfectly installed and
ports like ssh are filtered to outsiders. After you've played enough with
data, you can observe that you can submit jobs as well, and these jobs can
execute shell commands. Which is very, very sad.
In my view, this significantly limits distributed hadoop applications, where
part of your cluster may reside on EC2 or other distant datacenter, since
you always need to have certain ports open to an array of ip addresses (if
your instances are dynamic) which isn't acceptable if anyone from that ip
range can connect to your cluster.
Can we propose to developers to introduce some basic user-management and
access controls to help hadoop make one step further towards
production-quality system?
And, by the way add robots.txt to default distribution. (but I doubt it
will help, as it takes less than a week to scan all internet for given port
on home DSL..)
---
Dmitry
Re: Hadoop and security.
Posted by Ted Dunning <te...@gmail.com>.
Hadoopy is secure enough to be used on a cluster that has access control in
a friendly environment. That is to say, not very. These issues are well
known.
User identities were added recently, but, as you note, they are dependent on
trusting unix logins and can easily be spoofed. More secure identity
management is coming before too long.
In the EC2 environment, it is presumed that you would limit connectivity to
the hadoop cluster to just members of the cluster and machines that are
allowed to submit jobs using network security. This is essentially no
different from the way that things work with any application in EC2 such as
mySQL. You just don't allow external access.
Even with the new identity management, however, it isn't likely that you
would want to expose your hadoop cluster any more than you would want to
expose your database or nfs server. If outsiders need access, then you
should have a web tier that mediates that access.
On Sat, Oct 4, 2008 at 11:54 PM, Dmitry Pushkarev <um...@stanford.edu> wrote:
> Dear hadoop users,
>
>
>
> I'm lucky to work in academic environment where information security is not
> the question. However, I'm sure that most of the hadoop users aren't.
>
>
>
> Here is the question: how secure hadoop is? (or let's say foolproof)
>
>
>
> Here is the answer: http://www.google.com/search?client=opera
> <
> http://www.google.com/search?client=opera&rls=en&q=Hadoop+Map/Reduce+Admini
> stration&sourceid=opera&ie=utf-8&oe=utf-8<http://www.google.com/search?client=opera&rls=en&q=Hadoop+Map/Reduce+Administration&sourceid=opera&ie=utf-8&oe=utf-8>
> >
> &rls=en&q=Hadoop+Map/Reduce+Administration&sourceid=opera&ie=utf-8&oe=utf-8
> not quite.
>
>
>
> What we're seeing here is open hadoop cluster, where anyone who capable of
> installing hadoop and changing his username to webcrawl can use their
> cluster and read their data, even though firewall is perfectly installed
> and
> ports like ssh are filtered to outsiders. After you've played enough with
> data, you can observe that you can submit jobs as well, and these jobs can
> execute shell commands. Which is very, very sad.
>
>
>
> In my view, this significantly limits distributed hadoop applications,
> where
> part of your cluster may reside on EC2 or other distant datacenter, since
> you always need to have certain ports open to an array of ip addresses (if
> your instances are dynamic) which isn't acceptable if anyone from that ip
> range can connect to your cluster.
>
>
>
> Can we propose to developers to introduce some basic user-management and
> access controls to help hadoop make one step further towards
> production-quality system?
>
>
>
> And, by the way add robots.txt to default distribution. (but I doubt it
> will help, as it takes less than a week to scan all internet for given port
> on home DSL..)
>
>
>
> ---
>
> Dmitry
>
>
>
>
--
ted
Re: Hadoop and security.
Posted by Steve Loughran <st...@apache.org>.
Allen Wittenauer wrote:
>
>
> On 10/6/08 6:39 AM, "Steve Loughran" <st...@apache.org> wrote:
>
>> Edward Capriolo wrote:
>>> You bring up some valid points. This would be a great topic for a
>>> white paper.
>> -a wiki page would be a start too
>
> I was thinking about doing "Deploying Hadoop Securely" for a ApacheCon EU
> talk, as by that time, some of the basic Kerberos stuff should be in
> place... This whole conversation is starting to reinforce the idea....
>
>
-Start with an ApacheCon US fastfeather talk on the current state of
affairs "a hadoop cluster is as secure as a farm of machines running
NFS". Just to let people know where things stand.
for the EU one, I will probably put in for one on deploying/managing
using our toolset.
I'm also thinking of a talk "datamining a city" that looks at what data
sources a city is already instrumented with, if only you could get at
them. The hard part is getting at them. I have my eye on our local speed
camera/red light cameras, that track the speed of every vehicle passing
and time of day; you could build up a map of traffic velocity, where the
jams are, when, etc. Getting the machine-parsed number plate data would
be even more interesting, but governments tend to restrict that data to
state security, rather than useful things like analysing and predicting
traffic flow.
Re: Hadoop and security.
Posted by Allen Wittenauer <aw...@yahoo-inc.com>.
On 10/6/08 6:39 AM, "Steve Loughran" <st...@apache.org> wrote:
> Edward Capriolo wrote:
>> You bring up some valid points. This would be a great topic for a
>> white paper.
>
> -a wiki page would be a start too
I was thinking about doing "Deploying Hadoop Securely" for a ApacheCon EU
talk, as by that time, some of the basic Kerberos stuff should be in
place... This whole conversation is starting to reinforce the idea....
Re: Hadoop and security.
Posted by Steve Loughran <st...@apache.org>.
Edward Capriolo wrote:
> You bring up some valid points. This would be a great topic for a
> white paper.
-a wiki page would be a start too
The first line of defense should be to apply inbound and
> outbound iptables rules. Only source IPs that have a direct need to
> interact with the cluster should be allowed to. The same is true with
> the web access. Only a range of source IP's should be allowed to
> access the web interfaces. You can do this through SSH tunneling.
>
> Preventing exec commands can be handled with the security manager and
> the sandbox. I was thinking to only allow the execution of signed jars
> myself but I never implemented it.
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
Re: Hadoop and security.
Posted by Edward Capriolo <ed...@gmail.com>.
You bring up some valid points. This would be a great topic for a
white paper. The first line of defense should be to apply inbound and
outbound iptables rules. Only source IPs that have a direct need to
interact with the cluster should be allowed to. The same is true with
the web access. Only a range of source IP's should be allowed to
access the web interfaces. You can do this through SSH tunneling.
Preventing exec commands can be handled with the security manager and
the sandbox. I was thinking to only allow the execution of signed jars
myself but I never implemented it.
Re: Hadoop and security.
Posted by Steve Loughran <st...@apache.org>.
Dmitry Pushkarev wrote:
> Dear hadoop users,
>
>
>
> I'm lucky to work in academic environment where information security is not
> the question. However, I'm sure that most of the hadoop users aren't.
>
>
>
> Here is the question: how secure hadoop is? (or let's say foolproof)
Right now hadoop is about as secure as NFS. when deployed onto private
datacentres with good physical security and well set up networks, you
can control who gets at the data. Without that, you are sharing your
state with anyone who can issue HTTP and hadoop IPC requests.
>
>
>
> Here is the answer: http://www.google.com/search?client=opera
> <http://www.google.com/search?client=opera&rls=en&q=Hadoop+Map/Reduce+Admini
> stration&sourceid=opera&ie=utf-8&oe=utf-8>
> &rls=en&q=Hadoop+Map/Reduce+Administration&sourceid=opera&ie=utf-8&oe=utf-8
> not quite.
>
see also http://www.google.com/search?q=axis+happiness+page ; pages
that we add for benefit of the ops team end up sneaking out into the big
net.
>
>
> What we're seeing here is open hadoop cluster, where anyone who capable of
> installing hadoop and changing his username to webcrawl can use their
> cluster and read their data, even though firewall is perfectly installed and
> ports like ssh are filtered to outsiders. After you've played enough with
> data, you can observe that you can submit jobs as well, and these jobs can
> execute shell commands. Which is very, very sad.
>
>
>
> In my view, this significantly limits distributed hadoop applications, where
> part of your cluster may reside on EC2 or other distant datacenter, since
> you always need to have certain ports open to an array of ip addresses (if
> your instances are dynamic) which isn't acceptable if anyone from that ip
> range can connect to your cluster.
well, maybe that's a fault of EC2s architecture in which a deployment
request doesn't include a declaration of the network configuration?
>
>
>
> Can we propose to developers to introduce some basic user-management and
> access controls to help hadoop make one step further towards
> production-quality system?
>
Being an open source project, you can do more than propose, you can help
build some basic user-management and access controls. As to "production
quality"; it is ready for production, albeit in locked down datacentres.
Which is the primary deployment infrastructure of many of the active
developers. As in most community-contributed open source projects, if
you have specific needs beyond what the active developers need, you end
up implementing them your self.
The big issue with security is that it is all or nothing. Right now it
is blatantly insecure, so you should not be surprised that anyone has
access to your files. To actually lock it down, you would need to
authenticate and possibly encrypt all communications; this adds a lot of
overhead, which is why it will be avoided in the big datacentres. You
also need to go to a lot of effort to make sure it is secure across the
board, with no JSP pages providing accidental privilege escalation, no
api calls letting you see stuff you shouldn't. Its not like a normal
feature defect where you can say "don't do that"; it's not so easy to
validate using functional tests that test the expected uses of the code.
This is why securing an application is such a hard thing to do.
--
Steve Loughran http://www.1060.org/blogxter/publish/5
Author: Ant in Action http://antbook.org/
Re: Hadoop and security.
Posted by Allen Wittenauer <aw...@yahoo-inc.com>.
On 10/4/08 11:54 PM, "Dmitry Pushkarev" <um...@stanford.edu> wrote:
> Here is the question: how secure hadoop is? (or let's say foolproof)
I'd hope it was fairly well known that Hadoop is not secure. At all.
> What we're seeing here is open hadoop cluster, where anyone who capable of
> installing hadoop and changing his username to webcrawl can use their
> cluster and read their data, even though firewall is perfectly installed and
> ports like ssh are filtered to outsiders.
The firewall is likely blocking everything <1024, but making the (bad)
assumption that anything in the ephemeral range is 'safe'. A common
misconfiguration.
In any case, the hadoop ports should be blocked to the outside world.
> Can we propose to developers to introduce some basic user-management and
> access controls to help hadoop make one step further towards
> production-quality system?
https://issues.apache.org/jira/browse/HADOOP-1701 ,
https://issues.apache.org/jira/browse/HADOOP-1741 ,
https://issues.apache.org/jira/browse/HADOOP-3854 ,
https://issues.apache.org/jira/browse/HADOOP-4343 , ...
amongst others.
> And, by the way add robots.txt to default distribution.
https://issues.apache.org/jira/browse/HADOOP-3397