You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by Visioner Sadak <vi...@gmail.com> on 2013/08/30 08:35:26 UTC
authentication when uploading in to hadoop HDFS
Hello friends we use filesystem.copyFrmLocal method of java api within a
tomcat conntainer to move data in to hadoop clusters, will any other
unauthorised user will be able to write in to our hadoop cluster using the
java api or is any extra authenticaton needed from our side
Re: authentication when uploading in to hadoop HDFS
Posted by Larry McCay <lm...@hortonworks.com>.
Hi Visioner -
Depending on your actual installation, you may have all of the other APIs
available to the CLI clients as well.
This would potentially be an valid usecase for Apache Knox - in the
incubator still - see: http://knox.incubator.apache.org/
Knox provides you with a Web API Gateway for Hadoop. There is of course
support for webhdfs built into the gateway.
What this would allow you todo is wall off your Hadoop cluster with
appropriate networking techniques - such as firewalls - and only open the
Knox Gateway port to the network that you external users have access to.
You can then authenticate incoming REST requests using BASIC authentication
against LDAP or you can build a customer authentication provider for your
environment - if needed.
You would want to switch to the webhdfs API for moving files into HDFS
though.
I would encourage you to subscribe to the user/dev lists for Knox and start
a discussion there. We would be happy to help you with your web app access
there.
thanks,
--larry
On Fri, Aug 30, 2013 at 7:51 AM, Nitin Pawar <ni...@gmail.com>wrote:
> ssh has nothing to do with hdfs.
>
> there are three ways someone would want to write into hdfs
> 1) HDFS java api
> 2) hadoop command line tools
> 3) Webhdfs (doing post, put etc)
>
> In all above cases, there is no role of ssh. So you can assume that as
> long as no one has access to ssh-keys, no one can get into your hardware
> cluster. This does not mean that you have safe hdfs.
> To setup hdfs security you will need to
> 1) Isolate your cluster from public networks. (Even if your cluster has
> public ips, your network should only allows traffic from known addreses)
> 2) Default hdfs security is like POSIX systems, so you can check that
> 3) You really want to security then you can go for kerberos based
> authentications, do to anything on your cluster.
>
>
> Please wait for few experts to give you some ideas.
>
>
> On Fri, Aug 30, 2013 at 4:43 PM, Visioner Sadak <vi...@gmail.com>wrote:
>
>> Thanks a ton Nitin just wanted to confirm for the point below
>>
>> an external user wont be able to write in to our cluster using any API
>> right as we didnt included his ip in our cluster using password less ssh
>> for him i guess ssh will prompt a password for writes and reads correct me
>> if i am wrong :)
>>
>>
>> only admin has ssh access to linux clusters
>> ====>if no one has ssh access then password less ssh does not do any harm.
>>
>> On Fri, Aug 30, 2013 at 12:35 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>
>>> well have access to read from hdfs using webhdfs :
>>> ===>you may want to secure it with IP and username based authentications
>>>
>>> as of now we dunt have any security specific to hdfs user level we have
>>> se permissions=true for a particular user
>>> ====>if you are managing user level access control then it should be
>>> technically safe that anyone other that hdfs superuser can not create and
>>> change permissions for user directories.
>>>
>>> only admin has ssh access to linux clusters
>>> ====>if no one has ssh access then password less ssh does not do any
>>> harm.
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak <
>>> visioner.sadak@gmail.com> wrote:
>>>
>>>> well have access to read from hdfs using webhdfs
>>>>
>>>> as of now we dunt have any security specific to hdfs
>>>>
>>>> user level we have se permissions=true for a particular user
>>>>
>>>> only admin has ssh access to linux clusters
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>>>
>>>>> Visioner,
>>>>> is your cluster accessible on public network?
>>>>> What kind of hdfs security you have kept in place?
>>>>> what is your cluster security?(user level, intranet level)
>>>>> who all have ssh-keys to login to any node on the cluster?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>>>>> visioner.sadak@gmail.com> wrote:
>>>>>
>>>>>> also we have done a password-less ssh within our clusters only so
>>>>>> that we can access the cluster but i guess this wont be the case for an
>>>>>> external user right
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>>>>> visioner.sadak@gmail.com> wrote:
>>>>>>
>>>>>>> Hello friends we use filesystem.copyFrmLocal method of java
>>>>>>> api within a tomcat conntainer to move data in to hadoop clusters, will
>>>>>>> any other unauthorised user will be able to write in to our hadoop cluster
>>>>>>> using the java api or is any extra authenticaton needed from our side
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Nitin Pawar
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
>
> --
> Nitin Pawar
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: authentication when uploading in to hadoop HDFS
Posted by Larry McCay <lm...@hortonworks.com>.
Hi Visioner -
Depending on your actual installation, you may have all of the other APIs
available to the CLI clients as well.
This would potentially be an valid usecase for Apache Knox - in the
incubator still - see: http://knox.incubator.apache.org/
Knox provides you with a Web API Gateway for Hadoop. There is of course
support for webhdfs built into the gateway.
What this would allow you todo is wall off your Hadoop cluster with
appropriate networking techniques - such as firewalls - and only open the
Knox Gateway port to the network that you external users have access to.
You can then authenticate incoming REST requests using BASIC authentication
against LDAP or you can build a customer authentication provider for your
environment - if needed.
You would want to switch to the webhdfs API for moving files into HDFS
though.
I would encourage you to subscribe to the user/dev lists for Knox and start
a discussion there. We would be happy to help you with your web app access
there.
thanks,
--larry
On Fri, Aug 30, 2013 at 7:51 AM, Nitin Pawar <ni...@gmail.com>wrote:
> ssh has nothing to do with hdfs.
>
> there are three ways someone would want to write into hdfs
> 1) HDFS java api
> 2) hadoop command line tools
> 3) Webhdfs (doing post, put etc)
>
> In all above cases, there is no role of ssh. So you can assume that as
> long as no one has access to ssh-keys, no one can get into your hardware
> cluster. This does not mean that you have safe hdfs.
> To setup hdfs security you will need to
> 1) Isolate your cluster from public networks. (Even if your cluster has
> public ips, your network should only allows traffic from known addreses)
> 2) Default hdfs security is like POSIX systems, so you can check that
> 3) You really want to security then you can go for kerberos based
> authentications, do to anything on your cluster.
>
>
> Please wait for few experts to give you some ideas.
>
>
> On Fri, Aug 30, 2013 at 4:43 PM, Visioner Sadak <vi...@gmail.com>wrote:
>
>> Thanks a ton Nitin just wanted to confirm for the point below
>>
>> an external user wont be able to write in to our cluster using any API
>> right as we didnt included his ip in our cluster using password less ssh
>> for him i guess ssh will prompt a password for writes and reads correct me
>> if i am wrong :)
>>
>>
>> only admin has ssh access to linux clusters
>> ====>if no one has ssh access then password less ssh does not do any harm.
>>
>> On Fri, Aug 30, 2013 at 12:35 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>
>>> well have access to read from hdfs using webhdfs :
>>> ===>you may want to secure it with IP and username based authentications
>>>
>>> as of now we dunt have any security specific to hdfs user level we have
>>> se permissions=true for a particular user
>>> ====>if you are managing user level access control then it should be
>>> technically safe that anyone other that hdfs superuser can not create and
>>> change permissions for user directories.
>>>
>>> only admin has ssh access to linux clusters
>>> ====>if no one has ssh access then password less ssh does not do any
>>> harm.
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak <
>>> visioner.sadak@gmail.com> wrote:
>>>
>>>> well have access to read from hdfs using webhdfs
>>>>
>>>> as of now we dunt have any security specific to hdfs
>>>>
>>>> user level we have se permissions=true for a particular user
>>>>
>>>> only admin has ssh access to linux clusters
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>>>
>>>>> Visioner,
>>>>> is your cluster accessible on public network?
>>>>> What kind of hdfs security you have kept in place?
>>>>> what is your cluster security?(user level, intranet level)
>>>>> who all have ssh-keys to login to any node on the cluster?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>>>>> visioner.sadak@gmail.com> wrote:
>>>>>
>>>>>> also we have done a password-less ssh within our clusters only so
>>>>>> that we can access the cluster but i guess this wont be the case for an
>>>>>> external user right
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>>>>> visioner.sadak@gmail.com> wrote:
>>>>>>
>>>>>>> Hello friends we use filesystem.copyFrmLocal method of java
>>>>>>> api within a tomcat conntainer to move data in to hadoop clusters, will
>>>>>>> any other unauthorised user will be able to write in to our hadoop cluster
>>>>>>> using the java api or is any extra authenticaton needed from our side
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Nitin Pawar
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
>
> --
> Nitin Pawar
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: authentication when uploading in to hadoop HDFS
Posted by Larry McCay <lm...@hortonworks.com>.
Hi Visioner -
Depending on your actual installation, you may have all of the other APIs
available to the CLI clients as well.
This would potentially be an valid usecase for Apache Knox - in the
incubator still - see: http://knox.incubator.apache.org/
Knox provides you with a Web API Gateway for Hadoop. There is of course
support for webhdfs built into the gateway.
What this would allow you todo is wall off your Hadoop cluster with
appropriate networking techniques - such as firewalls - and only open the
Knox Gateway port to the network that you external users have access to.
You can then authenticate incoming REST requests using BASIC authentication
against LDAP or you can build a customer authentication provider for your
environment - if needed.
You would want to switch to the webhdfs API for moving files into HDFS
though.
I would encourage you to subscribe to the user/dev lists for Knox and start
a discussion there. We would be happy to help you with your web app access
there.
thanks,
--larry
On Fri, Aug 30, 2013 at 7:51 AM, Nitin Pawar <ni...@gmail.com>wrote:
> ssh has nothing to do with hdfs.
>
> there are three ways someone would want to write into hdfs
> 1) HDFS java api
> 2) hadoop command line tools
> 3) Webhdfs (doing post, put etc)
>
> In all above cases, there is no role of ssh. So you can assume that as
> long as no one has access to ssh-keys, no one can get into your hardware
> cluster. This does not mean that you have safe hdfs.
> To setup hdfs security you will need to
> 1) Isolate your cluster from public networks. (Even if your cluster has
> public ips, your network should only allows traffic from known addreses)
> 2) Default hdfs security is like POSIX systems, so you can check that
> 3) You really want to security then you can go for kerberos based
> authentications, do to anything on your cluster.
>
>
> Please wait for few experts to give you some ideas.
>
>
> On Fri, Aug 30, 2013 at 4:43 PM, Visioner Sadak <vi...@gmail.com>wrote:
>
>> Thanks a ton Nitin just wanted to confirm for the point below
>>
>> an external user wont be able to write in to our cluster using any API
>> right as we didnt included his ip in our cluster using password less ssh
>> for him i guess ssh will prompt a password for writes and reads correct me
>> if i am wrong :)
>>
>>
>> only admin has ssh access to linux clusters
>> ====>if no one has ssh access then password less ssh does not do any harm.
>>
>> On Fri, Aug 30, 2013 at 12:35 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>
>>> well have access to read from hdfs using webhdfs :
>>> ===>you may want to secure it with IP and username based authentications
>>>
>>> as of now we dunt have any security specific to hdfs user level we have
>>> se permissions=true for a particular user
>>> ====>if you are managing user level access control then it should be
>>> technically safe that anyone other that hdfs superuser can not create and
>>> change permissions for user directories.
>>>
>>> only admin has ssh access to linux clusters
>>> ====>if no one has ssh access then password less ssh does not do any
>>> harm.
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak <
>>> visioner.sadak@gmail.com> wrote:
>>>
>>>> well have access to read from hdfs using webhdfs
>>>>
>>>> as of now we dunt have any security specific to hdfs
>>>>
>>>> user level we have se permissions=true for a particular user
>>>>
>>>> only admin has ssh access to linux clusters
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>>>
>>>>> Visioner,
>>>>> is your cluster accessible on public network?
>>>>> What kind of hdfs security you have kept in place?
>>>>> what is your cluster security?(user level, intranet level)
>>>>> who all have ssh-keys to login to any node on the cluster?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>>>>> visioner.sadak@gmail.com> wrote:
>>>>>
>>>>>> also we have done a password-less ssh within our clusters only so
>>>>>> that we can access the cluster but i guess this wont be the case for an
>>>>>> external user right
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>>>>> visioner.sadak@gmail.com> wrote:
>>>>>>
>>>>>>> Hello friends we use filesystem.copyFrmLocal method of java
>>>>>>> api within a tomcat conntainer to move data in to hadoop clusters, will
>>>>>>> any other unauthorised user will be able to write in to our hadoop cluster
>>>>>>> using the java api or is any extra authenticaton needed from our side
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Nitin Pawar
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
>
> --
> Nitin Pawar
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: authentication when uploading in to hadoop HDFS
Posted by Larry McCay <lm...@hortonworks.com>.
Hi Visioner -
Depending on your actual installation, you may have all of the other APIs
available to the CLI clients as well.
This would potentially be an valid usecase for Apache Knox - in the
incubator still - see: http://knox.incubator.apache.org/
Knox provides you with a Web API Gateway for Hadoop. There is of course
support for webhdfs built into the gateway.
What this would allow you todo is wall off your Hadoop cluster with
appropriate networking techniques - such as firewalls - and only open the
Knox Gateway port to the network that you external users have access to.
You can then authenticate incoming REST requests using BASIC authentication
against LDAP or you can build a customer authentication provider for your
environment - if needed.
You would want to switch to the webhdfs API for moving files into HDFS
though.
I would encourage you to subscribe to the user/dev lists for Knox and start
a discussion there. We would be happy to help you with your web app access
there.
thanks,
--larry
On Fri, Aug 30, 2013 at 7:51 AM, Nitin Pawar <ni...@gmail.com>wrote:
> ssh has nothing to do with hdfs.
>
> there are three ways someone would want to write into hdfs
> 1) HDFS java api
> 2) hadoop command line tools
> 3) Webhdfs (doing post, put etc)
>
> In all above cases, there is no role of ssh. So you can assume that as
> long as no one has access to ssh-keys, no one can get into your hardware
> cluster. This does not mean that you have safe hdfs.
> To setup hdfs security you will need to
> 1) Isolate your cluster from public networks. (Even if your cluster has
> public ips, your network should only allows traffic from known addreses)
> 2) Default hdfs security is like POSIX systems, so you can check that
> 3) You really want to security then you can go for kerberos based
> authentications, do to anything on your cluster.
>
>
> Please wait for few experts to give you some ideas.
>
>
> On Fri, Aug 30, 2013 at 4:43 PM, Visioner Sadak <vi...@gmail.com>wrote:
>
>> Thanks a ton Nitin just wanted to confirm for the point below
>>
>> an external user wont be able to write in to our cluster using any API
>> right as we didnt included his ip in our cluster using password less ssh
>> for him i guess ssh will prompt a password for writes and reads correct me
>> if i am wrong :)
>>
>>
>> only admin has ssh access to linux clusters
>> ====>if no one has ssh access then password less ssh does not do any harm.
>>
>> On Fri, Aug 30, 2013 at 12:35 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>
>>> well have access to read from hdfs using webhdfs :
>>> ===>you may want to secure it with IP and username based authentications
>>>
>>> as of now we dunt have any security specific to hdfs user level we have
>>> se permissions=true for a particular user
>>> ====>if you are managing user level access control then it should be
>>> technically safe that anyone other that hdfs superuser can not create and
>>> change permissions for user directories.
>>>
>>> only admin has ssh access to linux clusters
>>> ====>if no one has ssh access then password less ssh does not do any
>>> harm.
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak <
>>> visioner.sadak@gmail.com> wrote:
>>>
>>>> well have access to read from hdfs using webhdfs
>>>>
>>>> as of now we dunt have any security specific to hdfs
>>>>
>>>> user level we have se permissions=true for a particular user
>>>>
>>>> only admin has ssh access to linux clusters
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>>>
>>>>> Visioner,
>>>>> is your cluster accessible on public network?
>>>>> What kind of hdfs security you have kept in place?
>>>>> what is your cluster security?(user level, intranet level)
>>>>> who all have ssh-keys to login to any node on the cluster?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>>>>> visioner.sadak@gmail.com> wrote:
>>>>>
>>>>>> also we have done a password-less ssh within our clusters only so
>>>>>> that we can access the cluster but i guess this wont be the case for an
>>>>>> external user right
>>>>>>
>>>>>>
>>>>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>>>>> visioner.sadak@gmail.com> wrote:
>>>>>>
>>>>>>> Hello friends we use filesystem.copyFrmLocal method of java
>>>>>>> api within a tomcat conntainer to move data in to hadoop clusters, will
>>>>>>> any other unauthorised user will be able to write in to our hadoop cluster
>>>>>>> using the java api or is any extra authenticaton needed from our side
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Nitin Pawar
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
>
> --
> Nitin Pawar
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: authentication when uploading in to hadoop HDFS
Posted by Nitin Pawar <ni...@gmail.com>.
ssh has nothing to do with hdfs.
there are three ways someone would want to write into hdfs
1) HDFS java api
2) hadoop command line tools
3) Webhdfs (doing post, put etc)
In all above cases, there is no role of ssh. So you can assume that as long
as no one has access to ssh-keys, no one can get into your hardware
cluster. This does not mean that you have safe hdfs.
To setup hdfs security you will need to
1) Isolate your cluster from public networks. (Even if your cluster has
public ips, your network should only allows traffic from known addreses)
2) Default hdfs security is like POSIX systems, so you can check that
3) You really want to security then you can go for kerberos based
authentications, do to anything on your cluster.
Please wait for few experts to give you some ideas.
On Fri, Aug 30, 2013 at 4:43 PM, Visioner Sadak <vi...@gmail.com>wrote:
> Thanks a ton Nitin just wanted to confirm for the point below
>
> an external user wont be able to write in to our cluster using any API
> right as we didnt included his ip in our cluster using password less ssh
> for him i guess ssh will prompt a password for writes and reads correct me
> if i am wrong :)
>
>
> only admin has ssh access to linux clusters
> ====>if no one has ssh access then password less ssh does not do any harm.
>
> On Fri, Aug 30, 2013 at 12:35 PM, Nitin Pawar <ni...@gmail.com>wrote:
>
>> well have access to read from hdfs using webhdfs :
>> ===>you may want to secure it with IP and username based authentications
>>
>> as of now we dunt have any security specific to hdfs user level we have
>> se permissions=true for a particular user
>> ====>if you are managing user level access control then it should be
>> technically safe that anyone other that hdfs superuser can not create and
>> change permissions for user directories.
>>
>> only admin has ssh access to linux clusters
>> ====>if no one has ssh access then password less ssh does not do any harm.
>>
>>
>> On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak <
>> visioner.sadak@gmail.com> wrote:
>>
>>> well have access to read from hdfs using webhdfs
>>>
>>> as of now we dunt have any security specific to hdfs
>>>
>>> user level we have se permissions=true for a particular user
>>>
>>> only admin has ssh access to linux clusters
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>>
>>>> Visioner,
>>>> is your cluster accessible on public network?
>>>> What kind of hdfs security you have kept in place?
>>>> what is your cluster security?(user level, intranet level)
>>>> who all have ssh-keys to login to any node on the cluster?
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>>>> visioner.sadak@gmail.com> wrote:
>>>>
>>>>> also we have done a password-less ssh within our clusters only so
>>>>> that we can access the cluster but i guess this wont be the case for an
>>>>> external user right
>>>>>
>>>>>
>>>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>>>> visioner.sadak@gmail.com> wrote:
>>>>>
>>>>>> Hello friends we use filesystem.copyFrmLocal method of java
>>>>>> api within a tomcat conntainer to move data in to hadoop clusters, will
>>>>>> any other unauthorised user will be able to write in to our hadoop cluster
>>>>>> using the java api or is any extra authenticaton needed from our side
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Nitin Pawar
>>>>
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
--
Nitin Pawar
Re: authentication when uploading in to hadoop HDFS
Posted by Nitin Pawar <ni...@gmail.com>.
ssh has nothing to do with hdfs.
there are three ways someone would want to write into hdfs
1) HDFS java api
2) hadoop command line tools
3) Webhdfs (doing post, put etc)
In all above cases, there is no role of ssh. So you can assume that as long
as no one has access to ssh-keys, no one can get into your hardware
cluster. This does not mean that you have safe hdfs.
To setup hdfs security you will need to
1) Isolate your cluster from public networks. (Even if your cluster has
public ips, your network should only allows traffic from known addreses)
2) Default hdfs security is like POSIX systems, so you can check that
3) You really want to security then you can go for kerberos based
authentications, do to anything on your cluster.
Please wait for few experts to give you some ideas.
On Fri, Aug 30, 2013 at 4:43 PM, Visioner Sadak <vi...@gmail.com>wrote:
> Thanks a ton Nitin just wanted to confirm for the point below
>
> an external user wont be able to write in to our cluster using any API
> right as we didnt included his ip in our cluster using password less ssh
> for him i guess ssh will prompt a password for writes and reads correct me
> if i am wrong :)
>
>
> only admin has ssh access to linux clusters
> ====>if no one has ssh access then password less ssh does not do any harm.
>
> On Fri, Aug 30, 2013 at 12:35 PM, Nitin Pawar <ni...@gmail.com>wrote:
>
>> well have access to read from hdfs using webhdfs :
>> ===>you may want to secure it with IP and username based authentications
>>
>> as of now we dunt have any security specific to hdfs user level we have
>> se permissions=true for a particular user
>> ====>if you are managing user level access control then it should be
>> technically safe that anyone other that hdfs superuser can not create and
>> change permissions for user directories.
>>
>> only admin has ssh access to linux clusters
>> ====>if no one has ssh access then password less ssh does not do any harm.
>>
>>
>> On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak <
>> visioner.sadak@gmail.com> wrote:
>>
>>> well have access to read from hdfs using webhdfs
>>>
>>> as of now we dunt have any security specific to hdfs
>>>
>>> user level we have se permissions=true for a particular user
>>>
>>> only admin has ssh access to linux clusters
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>>
>>>> Visioner,
>>>> is your cluster accessible on public network?
>>>> What kind of hdfs security you have kept in place?
>>>> what is your cluster security?(user level, intranet level)
>>>> who all have ssh-keys to login to any node on the cluster?
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>>>> visioner.sadak@gmail.com> wrote:
>>>>
>>>>> also we have done a password-less ssh within our clusters only so
>>>>> that we can access the cluster but i guess this wont be the case for an
>>>>> external user right
>>>>>
>>>>>
>>>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>>>> visioner.sadak@gmail.com> wrote:
>>>>>
>>>>>> Hello friends we use filesystem.copyFrmLocal method of java
>>>>>> api within a tomcat conntainer to move data in to hadoop clusters, will
>>>>>> any other unauthorised user will be able to write in to our hadoop cluster
>>>>>> using the java api or is any extra authenticaton needed from our side
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Nitin Pawar
>>>>
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
--
Nitin Pawar
Re: authentication when uploading in to hadoop HDFS
Posted by Nitin Pawar <ni...@gmail.com>.
ssh has nothing to do with hdfs.
there are three ways someone would want to write into hdfs
1) HDFS java api
2) hadoop command line tools
3) Webhdfs (doing post, put etc)
In all above cases, there is no role of ssh. So you can assume that as long
as no one has access to ssh-keys, no one can get into your hardware
cluster. This does not mean that you have safe hdfs.
To setup hdfs security you will need to
1) Isolate your cluster from public networks. (Even if your cluster has
public ips, your network should only allows traffic from known addreses)
2) Default hdfs security is like POSIX systems, so you can check that
3) You really want to security then you can go for kerberos based
authentications, do to anything on your cluster.
Please wait for few experts to give you some ideas.
On Fri, Aug 30, 2013 at 4:43 PM, Visioner Sadak <vi...@gmail.com>wrote:
> Thanks a ton Nitin just wanted to confirm for the point below
>
> an external user wont be able to write in to our cluster using any API
> right as we didnt included his ip in our cluster using password less ssh
> for him i guess ssh will prompt a password for writes and reads correct me
> if i am wrong :)
>
>
> only admin has ssh access to linux clusters
> ====>if no one has ssh access then password less ssh does not do any harm.
>
> On Fri, Aug 30, 2013 at 12:35 PM, Nitin Pawar <ni...@gmail.com>wrote:
>
>> well have access to read from hdfs using webhdfs :
>> ===>you may want to secure it with IP and username based authentications
>>
>> as of now we dunt have any security specific to hdfs user level we have
>> se permissions=true for a particular user
>> ====>if you are managing user level access control then it should be
>> technically safe that anyone other that hdfs superuser can not create and
>> change permissions for user directories.
>>
>> only admin has ssh access to linux clusters
>> ====>if no one has ssh access then password less ssh does not do any harm.
>>
>>
>> On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak <
>> visioner.sadak@gmail.com> wrote:
>>
>>> well have access to read from hdfs using webhdfs
>>>
>>> as of now we dunt have any security specific to hdfs
>>>
>>> user level we have se permissions=true for a particular user
>>>
>>> only admin has ssh access to linux clusters
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>>
>>>> Visioner,
>>>> is your cluster accessible on public network?
>>>> What kind of hdfs security you have kept in place?
>>>> what is your cluster security?(user level, intranet level)
>>>> who all have ssh-keys to login to any node on the cluster?
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>>>> visioner.sadak@gmail.com> wrote:
>>>>
>>>>> also we have done a password-less ssh within our clusters only so
>>>>> that we can access the cluster but i guess this wont be the case for an
>>>>> external user right
>>>>>
>>>>>
>>>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>>>> visioner.sadak@gmail.com> wrote:
>>>>>
>>>>>> Hello friends we use filesystem.copyFrmLocal method of java
>>>>>> api within a tomcat conntainer to move data in to hadoop clusters, will
>>>>>> any other unauthorised user will be able to write in to our hadoop cluster
>>>>>> using the java api or is any extra authenticaton needed from our side
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Nitin Pawar
>>>>
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
--
Nitin Pawar
Re: authentication when uploading in to hadoop HDFS
Posted by Nitin Pawar <ni...@gmail.com>.
ssh has nothing to do with hdfs.
there are three ways someone would want to write into hdfs
1) HDFS java api
2) hadoop command line tools
3) Webhdfs (doing post, put etc)
In all above cases, there is no role of ssh. So you can assume that as long
as no one has access to ssh-keys, no one can get into your hardware
cluster. This does not mean that you have safe hdfs.
To setup hdfs security you will need to
1) Isolate your cluster from public networks. (Even if your cluster has
public ips, your network should only allows traffic from known addreses)
2) Default hdfs security is like POSIX systems, so you can check that
3) You really want to security then you can go for kerberos based
authentications, do to anything on your cluster.
Please wait for few experts to give you some ideas.
On Fri, Aug 30, 2013 at 4:43 PM, Visioner Sadak <vi...@gmail.com>wrote:
> Thanks a ton Nitin just wanted to confirm for the point below
>
> an external user wont be able to write in to our cluster using any API
> right as we didnt included his ip in our cluster using password less ssh
> for him i guess ssh will prompt a password for writes and reads correct me
> if i am wrong :)
>
>
> only admin has ssh access to linux clusters
> ====>if no one has ssh access then password less ssh does not do any harm.
>
> On Fri, Aug 30, 2013 at 12:35 PM, Nitin Pawar <ni...@gmail.com>wrote:
>
>> well have access to read from hdfs using webhdfs :
>> ===>you may want to secure it with IP and username based authentications
>>
>> as of now we dunt have any security specific to hdfs user level we have
>> se permissions=true for a particular user
>> ====>if you are managing user level access control then it should be
>> technically safe that anyone other that hdfs superuser can not create and
>> change permissions for user directories.
>>
>> only admin has ssh access to linux clusters
>> ====>if no one has ssh access then password less ssh does not do any harm.
>>
>>
>> On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak <
>> visioner.sadak@gmail.com> wrote:
>>
>>> well have access to read from hdfs using webhdfs
>>>
>>> as of now we dunt have any security specific to hdfs
>>>
>>> user level we have se permissions=true for a particular user
>>>
>>> only admin has ssh access to linux clusters
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>>
>>>> Visioner,
>>>> is your cluster accessible on public network?
>>>> What kind of hdfs security you have kept in place?
>>>> what is your cluster security?(user level, intranet level)
>>>> who all have ssh-keys to login to any node on the cluster?
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>>>> visioner.sadak@gmail.com> wrote:
>>>>
>>>>> also we have done a password-less ssh within our clusters only so
>>>>> that we can access the cluster but i guess this wont be the case for an
>>>>> external user right
>>>>>
>>>>>
>>>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>>>> visioner.sadak@gmail.com> wrote:
>>>>>
>>>>>> Hello friends we use filesystem.copyFrmLocal method of java
>>>>>> api within a tomcat conntainer to move data in to hadoop clusters, will
>>>>>> any other unauthorised user will be able to write in to our hadoop cluster
>>>>>> using the java api or is any extra authenticaton needed from our side
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Nitin Pawar
>>>>
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
--
Nitin Pawar
Re: authentication when uploading in to hadoop HDFS
Posted by Visioner Sadak <vi...@gmail.com>.
Thanks a ton Nitin just wanted to confirm for the point below
an external user wont be able to write in to our cluster using any API
right as we didnt included his ip in our cluster using password less ssh
for him i guess ssh will prompt a password for writes and reads correct me
if i am wrong :)
only admin has ssh access to linux clusters
====>if no one has ssh access then password less ssh does not do any harm.
On Fri, Aug 30, 2013 at 12:35 PM, Nitin Pawar <ni...@gmail.com>wrote:
> well have access to read from hdfs using webhdfs :
> ===>you may want to secure it with IP and username based authentications
>
> as of now we dunt have any security specific to hdfs user level we have
> se permissions=true for a particular user
> ====>if you are managing user level access control then it should be
> technically safe that anyone other that hdfs superuser can not create and
> change permissions for user directories.
>
> only admin has ssh access to linux clusters
> ====>if no one has ssh access then password less ssh does not do any harm.
>
>
> On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak <visioner.sadak@gmail.com
> > wrote:
>
>> well have access to read from hdfs using webhdfs
>>
>> as of now we dunt have any security specific to hdfs
>>
>> user level we have se permissions=true for a particular user
>>
>> only admin has ssh access to linux clusters
>>
>>
>>
>>
>>
>>
>> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>
>>> Visioner,
>>> is your cluster accessible on public network?
>>> What kind of hdfs security you have kept in place?
>>> what is your cluster security?(user level, intranet level)
>>> who all have ssh-keys to login to any node on the cluster?
>>>
>>>
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>>> visioner.sadak@gmail.com> wrote:
>>>
>>>> also we have done a password-less ssh within our clusters only so that
>>>> we can access the cluster but i guess this wont be the case for an
>>>> external user right
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>>> visioner.sadak@gmail.com> wrote:
>>>>
>>>>> Hello friends we use filesystem.copyFrmLocal method of java api within
>>>>> a tomcat conntainer to move data in to hadoop clusters, will any other
>>>>> unauthorised user will be able to write in to our hadoop cluster using the
>>>>> java api or is any extra authenticaton needed from our side
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
>
> --
> Nitin Pawar
>
Re: authentication when uploading in to hadoop HDFS
Posted by Visioner Sadak <vi...@gmail.com>.
Thanks a ton Nitin just wanted to confirm for the point below
an external user wont be able to write in to our cluster using any API
right as we didnt included his ip in our cluster using password less ssh
for him i guess ssh will prompt a password for writes and reads correct me
if i am wrong :)
only admin has ssh access to linux clusters
====>if no one has ssh access then password less ssh does not do any harm.
On Fri, Aug 30, 2013 at 12:35 PM, Nitin Pawar <ni...@gmail.com>wrote:
> well have access to read from hdfs using webhdfs :
> ===>you may want to secure it with IP and username based authentications
>
> as of now we dunt have any security specific to hdfs user level we have
> se permissions=true for a particular user
> ====>if you are managing user level access control then it should be
> technically safe that anyone other that hdfs superuser can not create and
> change permissions for user directories.
>
> only admin has ssh access to linux clusters
> ====>if no one has ssh access then password less ssh does not do any harm.
>
>
> On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak <visioner.sadak@gmail.com
> > wrote:
>
>> well have access to read from hdfs using webhdfs
>>
>> as of now we dunt have any security specific to hdfs
>>
>> user level we have se permissions=true for a particular user
>>
>> only admin has ssh access to linux clusters
>>
>>
>>
>>
>>
>>
>> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>
>>> Visioner,
>>> is your cluster accessible on public network?
>>> What kind of hdfs security you have kept in place?
>>> what is your cluster security?(user level, intranet level)
>>> who all have ssh-keys to login to any node on the cluster?
>>>
>>>
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>>> visioner.sadak@gmail.com> wrote:
>>>
>>>> also we have done a password-less ssh within our clusters only so that
>>>> we can access the cluster but i guess this wont be the case for an
>>>> external user right
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>>> visioner.sadak@gmail.com> wrote:
>>>>
>>>>> Hello friends we use filesystem.copyFrmLocal method of java api within
>>>>> a tomcat conntainer to move data in to hadoop clusters, will any other
>>>>> unauthorised user will be able to write in to our hadoop cluster using the
>>>>> java api or is any extra authenticaton needed from our side
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
>
> --
> Nitin Pawar
>
Re: authentication when uploading in to hadoop HDFS
Posted by Visioner Sadak <vi...@gmail.com>.
Thanks a ton Nitin just wanted to confirm for the point below
an external user wont be able to write in to our cluster using any API
right as we didnt included his ip in our cluster using password less ssh
for him i guess ssh will prompt a password for writes and reads correct me
if i am wrong :)
only admin has ssh access to linux clusters
====>if no one has ssh access then password less ssh does not do any harm.
On Fri, Aug 30, 2013 at 12:35 PM, Nitin Pawar <ni...@gmail.com>wrote:
> well have access to read from hdfs using webhdfs :
> ===>you may want to secure it with IP and username based authentications
>
> as of now we dunt have any security specific to hdfs user level we have
> se permissions=true for a particular user
> ====>if you are managing user level access control then it should be
> technically safe that anyone other that hdfs superuser can not create and
> change permissions for user directories.
>
> only admin has ssh access to linux clusters
> ====>if no one has ssh access then password less ssh does not do any harm.
>
>
> On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak <visioner.sadak@gmail.com
> > wrote:
>
>> well have access to read from hdfs using webhdfs
>>
>> as of now we dunt have any security specific to hdfs
>>
>> user level we have se permissions=true for a particular user
>>
>> only admin has ssh access to linux clusters
>>
>>
>>
>>
>>
>>
>> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>
>>> Visioner,
>>> is your cluster accessible on public network?
>>> What kind of hdfs security you have kept in place?
>>> what is your cluster security?(user level, intranet level)
>>> who all have ssh-keys to login to any node on the cluster?
>>>
>>>
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>>> visioner.sadak@gmail.com> wrote:
>>>
>>>> also we have done a password-less ssh within our clusters only so that
>>>> we can access the cluster but i guess this wont be the case for an
>>>> external user right
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>>> visioner.sadak@gmail.com> wrote:
>>>>
>>>>> Hello friends we use filesystem.copyFrmLocal method of java api within
>>>>> a tomcat conntainer to move data in to hadoop clusters, will any other
>>>>> unauthorised user will be able to write in to our hadoop cluster using the
>>>>> java api or is any extra authenticaton needed from our side
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
>
> --
> Nitin Pawar
>
Re: authentication when uploading in to hadoop HDFS
Posted by Visioner Sadak <vi...@gmail.com>.
Thanks a ton Nitin just wanted to confirm for the point below
an external user wont be able to write in to our cluster using any API
right as we didnt included his ip in our cluster using password less ssh
for him i guess ssh will prompt a password for writes and reads correct me
if i am wrong :)
only admin has ssh access to linux clusters
====>if no one has ssh access then password less ssh does not do any harm.
On Fri, Aug 30, 2013 at 12:35 PM, Nitin Pawar <ni...@gmail.com>wrote:
> well have access to read from hdfs using webhdfs :
> ===>you may want to secure it with IP and username based authentications
>
> as of now we dunt have any security specific to hdfs user level we have
> se permissions=true for a particular user
> ====>if you are managing user level access control then it should be
> technically safe that anyone other that hdfs superuser can not create and
> change permissions for user directories.
>
> only admin has ssh access to linux clusters
> ====>if no one has ssh access then password less ssh does not do any harm.
>
>
> On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak <visioner.sadak@gmail.com
> > wrote:
>
>> well have access to read from hdfs using webhdfs
>>
>> as of now we dunt have any security specific to hdfs
>>
>> user level we have se permissions=true for a particular user
>>
>> only admin has ssh access to linux clusters
>>
>>
>>
>>
>>
>>
>> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>>
>>> Visioner,
>>> is your cluster accessible on public network?
>>> What kind of hdfs security you have kept in place?
>>> what is your cluster security?(user level, intranet level)
>>> who all have ssh-keys to login to any node on the cluster?
>>>
>>>
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>>> visioner.sadak@gmail.com> wrote:
>>>
>>>> also we have done a password-less ssh within our clusters only so that
>>>> we can access the cluster but i guess this wont be the case for an
>>>> external user right
>>>>
>>>>
>>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>>> visioner.sadak@gmail.com> wrote:
>>>>
>>>>> Hello friends we use filesystem.copyFrmLocal method of java api within
>>>>> a tomcat conntainer to move data in to hadoop clusters, will any other
>>>>> unauthorised user will be able to write in to our hadoop cluster using the
>>>>> java api or is any extra authenticaton needed from our side
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Nitin Pawar
>>>
>>
>>
>
>
> --
> Nitin Pawar
>
Re: authentication when uploading in to hadoop HDFS
Posted by Nitin Pawar <ni...@gmail.com>.
well have access to read from hdfs using webhdfs :
===>you may want to secure it with IP and username based authentications
as of now we dunt have any security specific to hdfs user level we have se
permissions=true for a particular user
====>if you are managing user level access control then it should be
technically safe that anyone other that hdfs superuser can not create and
change permissions for user directories.
only admin has ssh access to linux clusters
====>if no one has ssh access then password less ssh does not do any harm.
On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak
<vi...@gmail.com>wrote:
> well have access to read from hdfs using webhdfs
>
> as of now we dunt have any security specific to hdfs
>
> user level we have se permissions=true for a particular user
>
> only admin has ssh access to linux clusters
>
>
>
>
>
>
> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>
>> Visioner,
>> is your cluster accessible on public network?
>> What kind of hdfs security you have kept in place?
>> what is your cluster security?(user level, intranet level)
>> who all have ssh-keys to login to any node on the cluster?
>>
>>
>>
>>
>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>> visioner.sadak@gmail.com> wrote:
>>
>>> also we have done a password-less ssh within our clusters only so that
>>> we can access the cluster but i guess this wont be the case for an
>>> external user right
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>> visioner.sadak@gmail.com> wrote:
>>>
>>>> Hello friends we use filesystem.copyFrmLocal method of java api within
>>>> a tomcat conntainer to move data in to hadoop clusters, will any other
>>>> unauthorised user will be able to write in to our hadoop cluster using the
>>>> java api or is any extra authenticaton needed from our side
>>>>
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
--
Nitin Pawar
Re: authentication when uploading in to hadoop HDFS
Posted by Nitin Pawar <ni...@gmail.com>.
well have access to read from hdfs using webhdfs :
===>you may want to secure it with IP and username based authentications
as of now we dunt have any security specific to hdfs user level we have se
permissions=true for a particular user
====>if you are managing user level access control then it should be
technically safe that anyone other that hdfs superuser can not create and
change permissions for user directories.
only admin has ssh access to linux clusters
====>if no one has ssh access then password less ssh does not do any harm.
On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak
<vi...@gmail.com>wrote:
> well have access to read from hdfs using webhdfs
>
> as of now we dunt have any security specific to hdfs
>
> user level we have se permissions=true for a particular user
>
> only admin has ssh access to linux clusters
>
>
>
>
>
>
> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>
>> Visioner,
>> is your cluster accessible on public network?
>> What kind of hdfs security you have kept in place?
>> what is your cluster security?(user level, intranet level)
>> who all have ssh-keys to login to any node on the cluster?
>>
>>
>>
>>
>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>> visioner.sadak@gmail.com> wrote:
>>
>>> also we have done a password-less ssh within our clusters only so that
>>> we can access the cluster but i guess this wont be the case for an
>>> external user right
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>> visioner.sadak@gmail.com> wrote:
>>>
>>>> Hello friends we use filesystem.copyFrmLocal method of java api within
>>>> a tomcat conntainer to move data in to hadoop clusters, will any other
>>>> unauthorised user will be able to write in to our hadoop cluster using the
>>>> java api or is any extra authenticaton needed from our side
>>>>
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
--
Nitin Pawar
Re: authentication when uploading in to hadoop HDFS
Posted by Nitin Pawar <ni...@gmail.com>.
well have access to read from hdfs using webhdfs :
===>you may want to secure it with IP and username based authentications
as of now we dunt have any security specific to hdfs user level we have se
permissions=true for a particular user
====>if you are managing user level access control then it should be
technically safe that anyone other that hdfs superuser can not create and
change permissions for user directories.
only admin has ssh access to linux clusters
====>if no one has ssh access then password less ssh does not do any harm.
On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak
<vi...@gmail.com>wrote:
> well have access to read from hdfs using webhdfs
>
> as of now we dunt have any security specific to hdfs
>
> user level we have se permissions=true for a particular user
>
> only admin has ssh access to linux clusters
>
>
>
>
>
>
> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>
>> Visioner,
>> is your cluster accessible on public network?
>> What kind of hdfs security you have kept in place?
>> what is your cluster security?(user level, intranet level)
>> who all have ssh-keys to login to any node on the cluster?
>>
>>
>>
>>
>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>> visioner.sadak@gmail.com> wrote:
>>
>>> also we have done a password-less ssh within our clusters only so that
>>> we can access the cluster but i guess this wont be the case for an
>>> external user right
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>> visioner.sadak@gmail.com> wrote:
>>>
>>>> Hello friends we use filesystem.copyFrmLocal method of java api within
>>>> a tomcat conntainer to move data in to hadoop clusters, will any other
>>>> unauthorised user will be able to write in to our hadoop cluster using the
>>>> java api or is any extra authenticaton needed from our side
>>>>
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
--
Nitin Pawar
Re: authentication when uploading in to hadoop HDFS
Posted by Nitin Pawar <ni...@gmail.com>.
well have access to read from hdfs using webhdfs :
===>you may want to secure it with IP and username based authentications
as of now we dunt have any security specific to hdfs user level we have se
permissions=true for a particular user
====>if you are managing user level access control then it should be
technically safe that anyone other that hdfs superuser can not create and
change permissions for user directories.
only admin has ssh access to linux clusters
====>if no one has ssh access then password less ssh does not do any harm.
On Fri, Aug 30, 2013 at 12:17 PM, Visioner Sadak
<vi...@gmail.com>wrote:
> well have access to read from hdfs using webhdfs
>
> as of now we dunt have any security specific to hdfs
>
> user level we have se permissions=true for a particular user
>
> only admin has ssh access to linux clusters
>
>
>
>
>
>
> On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
>
>> Visioner,
>> is your cluster accessible on public network?
>> What kind of hdfs security you have kept in place?
>> what is your cluster security?(user level, intranet level)
>> who all have ssh-keys to login to any node on the cluster?
>>
>>
>>
>>
>> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <
>> visioner.sadak@gmail.com> wrote:
>>
>>> also we have done a password-less ssh within our clusters only so that
>>> we can access the cluster but i guess this wont be the case for an
>>> external user right
>>>
>>>
>>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>>> visioner.sadak@gmail.com> wrote:
>>>
>>>> Hello friends we use filesystem.copyFrmLocal method of java api within
>>>> a tomcat conntainer to move data in to hadoop clusters, will any other
>>>> unauthorised user will be able to write in to our hadoop cluster using the
>>>> java api or is any extra authenticaton needed from our side
>>>>
>>>
>>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
--
Nitin Pawar
Re: authentication when uploading in to hadoop HDFS
Posted by Visioner Sadak <vi...@gmail.com>.
well have access to read from hdfs using webhdfs
as of now we dunt have any security specific to hdfs
user level we have se permissions=true for a particular user
only admin has ssh access to linux clusters
On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
> Visioner,
> is your cluster accessible on public network?
> What kind of hdfs security you have kept in place?
> what is your cluster security?(user level, intranet level)
> who all have ssh-keys to login to any node on the cluster?
>
>
>
>
> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <visioner.sadak@gmail.com
> > wrote:
>
>> also we have done a password-less ssh within our clusters only so that
>> we can access the cluster but i guess this wont be the case for an
>> external user right
>>
>>
>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>> visioner.sadak@gmail.com> wrote:
>>
>>> Hello friends we use filesystem.copyFrmLocal method of java api within a
>>> tomcat conntainer to move data in to hadoop clusters, will any other
>>> unauthorised user will be able to write in to our hadoop cluster using the
>>> java api or is any extra authenticaton needed from our side
>>>
>>
>>
>
>
> --
> Nitin Pawar
>
Re: authentication when uploading in to hadoop HDFS
Posted by Visioner Sadak <vi...@gmail.com>.
well have access to read from hdfs using webhdfs
as of now we dunt have any security specific to hdfs
user level we have se permissions=true for a particular user
only admin has ssh access to linux clusters
On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
> Visioner,
> is your cluster accessible on public network?
> What kind of hdfs security you have kept in place?
> what is your cluster security?(user level, intranet level)
> who all have ssh-keys to login to any node on the cluster?
>
>
>
>
> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <visioner.sadak@gmail.com
> > wrote:
>
>> also we have done a password-less ssh within our clusters only so that
>> we can access the cluster but i guess this wont be the case for an
>> external user right
>>
>>
>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>> visioner.sadak@gmail.com> wrote:
>>
>>> Hello friends we use filesystem.copyFrmLocal method of java api within a
>>> tomcat conntainer to move data in to hadoop clusters, will any other
>>> unauthorised user will be able to write in to our hadoop cluster using the
>>> java api or is any extra authenticaton needed from our side
>>>
>>
>>
>
>
> --
> Nitin Pawar
>
Re: authentication when uploading in to hadoop HDFS
Posted by Visioner Sadak <vi...@gmail.com>.
well have access to read from hdfs using webhdfs
as of now we dunt have any security specific to hdfs
user level we have se permissions=true for a particular user
only admin has ssh access to linux clusters
On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
> Visioner,
> is your cluster accessible on public network?
> What kind of hdfs security you have kept in place?
> what is your cluster security?(user level, intranet level)
> who all have ssh-keys to login to any node on the cluster?
>
>
>
>
> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <visioner.sadak@gmail.com
> > wrote:
>
>> also we have done a password-less ssh within our clusters only so that
>> we can access the cluster but i guess this wont be the case for an
>> external user right
>>
>>
>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>> visioner.sadak@gmail.com> wrote:
>>
>>> Hello friends we use filesystem.copyFrmLocal method of java api within a
>>> tomcat conntainer to move data in to hadoop clusters, will any other
>>> unauthorised user will be able to write in to our hadoop cluster using the
>>> java api or is any extra authenticaton needed from our side
>>>
>>
>>
>
>
> --
> Nitin Pawar
>
Re: authentication when uploading in to hadoop HDFS
Posted by Visioner Sadak <vi...@gmail.com>.
well have access to read from hdfs using webhdfs
as of now we dunt have any security specific to hdfs
user level we have se permissions=true for a particular user
only admin has ssh access to linux clusters
On Fri, Aug 30, 2013 at 12:14 PM, Nitin Pawar <ni...@gmail.com>wrote:
> Visioner,
> is your cluster accessible on public network?
> What kind of hdfs security you have kept in place?
> what is your cluster security?(user level, intranet level)
> who all have ssh-keys to login to any node on the cluster?
>
>
>
>
> On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak <visioner.sadak@gmail.com
> > wrote:
>
>> also we have done a password-less ssh within our clusters only so that
>> we can access the cluster but i guess this wont be the case for an
>> external user right
>>
>>
>> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <
>> visioner.sadak@gmail.com> wrote:
>>
>>> Hello friends we use filesystem.copyFrmLocal method of java api within a
>>> tomcat conntainer to move data in to hadoop clusters, will any other
>>> unauthorised user will be able to write in to our hadoop cluster using the
>>> java api or is any extra authenticaton needed from our side
>>>
>>
>>
>
>
> --
> Nitin Pawar
>
Re: authentication when uploading in to hadoop HDFS
Posted by Nitin Pawar <ni...@gmail.com>.
Visioner,
is your cluster accessible on public network?
What kind of hdfs security you have kept in place?
what is your cluster security?(user level, intranet level)
who all have ssh-keys to login to any node on the cluster?
On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak
<vi...@gmail.com>wrote:
> also we have done a password-less ssh within our clusters only so that
> we can access the cluster but i guess this wont be the case for an
> external user right
>
>
> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <visioner.sadak@gmail.com
> > wrote:
>
>> Hello friends we use filesystem.copyFrmLocal method of java api within a
>> tomcat conntainer to move data in to hadoop clusters, will any other
>> unauthorised user will be able to write in to our hadoop cluster using the
>> java api or is any extra authenticaton needed from our side
>>
>
>
--
Nitin Pawar
Re: authentication when uploading in to hadoop HDFS
Posted by Nitin Pawar <ni...@gmail.com>.
Visioner,
is your cluster accessible on public network?
What kind of hdfs security you have kept in place?
what is your cluster security?(user level, intranet level)
who all have ssh-keys to login to any node on the cluster?
On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak
<vi...@gmail.com>wrote:
> also we have done a password-less ssh within our clusters only so that
> we can access the cluster but i guess this wont be the case for an
> external user right
>
>
> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <visioner.sadak@gmail.com
> > wrote:
>
>> Hello friends we use filesystem.copyFrmLocal method of java api within a
>> tomcat conntainer to move data in to hadoop clusters, will any other
>> unauthorised user will be able to write in to our hadoop cluster using the
>> java api or is any extra authenticaton needed from our side
>>
>
>
--
Nitin Pawar
Re: authentication when uploading in to hadoop HDFS
Posted by Nitin Pawar <ni...@gmail.com>.
Visioner,
is your cluster accessible on public network?
What kind of hdfs security you have kept in place?
what is your cluster security?(user level, intranet level)
who all have ssh-keys to login to any node on the cluster?
On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak
<vi...@gmail.com>wrote:
> also we have done a password-less ssh within our clusters only so that
> we can access the cluster but i guess this wont be the case for an
> external user right
>
>
> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <visioner.sadak@gmail.com
> > wrote:
>
>> Hello friends we use filesystem.copyFrmLocal method of java api within a
>> tomcat conntainer to move data in to hadoop clusters, will any other
>> unauthorised user will be able to write in to our hadoop cluster using the
>> java api or is any extra authenticaton needed from our side
>>
>
>
--
Nitin Pawar
Re: authentication when uploading in to hadoop HDFS
Posted by Nitin Pawar <ni...@gmail.com>.
Visioner,
is your cluster accessible on public network?
What kind of hdfs security you have kept in place?
what is your cluster security?(user level, intranet level)
who all have ssh-keys to login to any node on the cluster?
On Fri, Aug 30, 2013 at 12:08 PM, Visioner Sadak
<vi...@gmail.com>wrote:
> also we have done a password-less ssh within our clusters only so that
> we can access the cluster but i guess this wont be the case for an
> external user right
>
>
> On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak <visioner.sadak@gmail.com
> > wrote:
>
>> Hello friends we use filesystem.copyFrmLocal method of java api within a
>> tomcat conntainer to move data in to hadoop clusters, will any other
>> unauthorised user will be able to write in to our hadoop cluster using the
>> java api or is any extra authenticaton needed from our side
>>
>
>
--
Nitin Pawar
Re: authentication when uploading in to hadoop HDFS
Posted by Visioner Sadak <vi...@gmail.com>.
also we have done a password-less ssh within our clusters only so that we
can access the cluster but i guess this wont be the case for an external
user right
On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak
<vi...@gmail.com>wrote:
> Hello friends we use filesystem.copyFrmLocal method of java api within a
> tomcat conntainer to move data in to hadoop clusters, will any other
> unauthorised user will be able to write in to our hadoop cluster using the
> java api or is any extra authenticaton needed from our side
>
Re: authentication when uploading in to hadoop HDFS
Posted by Visioner Sadak <vi...@gmail.com>.
also we have done a password-less ssh within our clusters only so that we
can access the cluster but i guess this wont be the case for an external
user right
On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak
<vi...@gmail.com>wrote:
> Hello friends we use filesystem.copyFrmLocal method of java api within a
> tomcat conntainer to move data in to hadoop clusters, will any other
> unauthorised user will be able to write in to our hadoop cluster using the
> java api or is any extra authenticaton needed from our side
>
Re: authentication when uploading in to hadoop HDFS
Posted by Visioner Sadak <vi...@gmail.com>.
also we have done a password-less ssh within our clusters only so that we
can access the cluster but i guess this wont be the case for an external
user right
On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak
<vi...@gmail.com>wrote:
> Hello friends we use filesystem.copyFrmLocal method of java api within a
> tomcat conntainer to move data in to hadoop clusters, will any other
> unauthorised user will be able to write in to our hadoop cluster using the
> java api or is any extra authenticaton needed from our side
>
Re: authentication when uploading in to hadoop HDFS
Posted by Visioner Sadak <vi...@gmail.com>.
also we have done a password-less ssh within our clusters only so that we
can access the cluster but i guess this wont be the case for an external
user right
On Fri, Aug 30, 2013 at 12:05 PM, Visioner Sadak
<vi...@gmail.com>wrote:
> Hello friends we use filesystem.copyFrmLocal method of java api within a
> tomcat conntainer to move data in to hadoop clusters, will any other
> unauthorised user will be able to write in to our hadoop cluster using the
> java api or is any extra authenticaton needed from our side
>