You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Robert Irribarren <ro...@algorithms.io> on 2012/08/19 03:43:51 UTC
Permission denied hadoop job
bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
-copyFromLocal /tmp/myDir /user/bitnami/myDir
Warning: $HADOOP_HOME is deprecated.
copyFromLocal: Target /user/bitnami/myDir/ is a directory
bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs -rmr
/user/bitnami/
Warning: $HADOOP_HOME is deprecated.
rmr: org.apache.hadoop.security.AccessControlException: Permission denied:
user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
I have also made sure that bitnami was in the group hadoop
bitnami@ip-10-xxx-xx-xx:groups bitnami
bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev netdev
admin hadoop
I have read that the root:supergroup: should read bitnami:supergroup. How
can I change this?
Re: Permission denied hadoop job
Posted by Bertrand Dechoux <de...@gmail.com>.
It wasn't meant to be a sardonic retort and I apologize if it did sound
like it.
A few times, people ask questions which make think that they didn't have
access to the right documentation.
I believe I provided you enough information about your specific issue and a
way to understand better what you did right or wrong.
About security, without Kerberos there is no strong authentification and as
Hadoop does not warn (as I far as I know) against the superuser being the
system root, a few times people may believe there isn't any security and
thus may not understand the error thrown by the system. Which is why
knowing about any official documentation on the subject is important.
Regards
Bertrand
On Sun, Aug 19, 2012 at 8:34 AM, Robert Irribarren <ro...@algorithms.io>wrote:
> Thanks for the sardonic retort, I got it working. I just had to redo the
> steps and I set the hdfs to my tmp folder and had permissions set up
> properly
>
>
> On Sat, Aug 18, 2012 at 8:50 PM, Bertrand Dechoux <de...@gmail.com>wrote:
>
>> Hi,
>>
>> I have never heard about a magic group called 'hadoop'.
>> Does it have a specific meaning in your context?
>>
>> You should read more about security in Hadoop.
>> Your last question hint that you haven't looked at it.
>>
>>
>> http://hadoop.apache.org/common/docs/r1.0.3/hdfs_user_guide.html#File+Permissions+and+Security
>>
>> File Permissions and Security
>>> The file permissions are designed to be similar to file permissions on
>>> other familiar platforms like Linux. Currently, security is limited to
>>> simple file permissions. *The user that starts NameNode is treated as
>>> the superuser for HDFS.* Future versions of HDFS will support network
>>> authentication protocols like Kerberos for user authentication and
>>> encryption of data transfers. The details are discussed in the Permissions
>>> Guide<http://hadoop.apache.org/common/docs/r1.0.3/hdfs_permissions_guide.html>
>>> .
>>
>>
>> The straight answer is : run your cluster with the bitnami user.
>>
>> But it may be a good habit to
>> 1) have the hadoop superuser being different than the system root
>> 2) interact with your cluster with users having limited access ie not
>> your superuser
>>
>> Regards
>>
>> Bertrand
>>
>> On Sun, Aug 19, 2012 at 3:43 AM, Robert Irribarren <ro...@algorithms.io>wrote:
>>
>>> bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>>> -copyFromLocal /tmp/myDir /user/bitnami/myDir
>>> Warning: $HADOOP_HOME is deprecated.
>>>
>>> copyFromLocal: Target /user/bitnami/myDir/ is a directory
>>> bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>>> -rmr /user/bitnami/
>>> Warning: $HADOOP_HOME is deprecated.
>>>
>>> rmr: org.apache.hadoop.security.AccessControlException: Permission
>>> denied: user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
>>>
>>>
>>> I have also made sure that bitnami was in the group hadoop
>>>
>>> bitnami@ip-10-xxx-xx-xx:groups bitnami
>>> bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev
>>> netdev admin hadoop
>>>
>>> I have read that the root:supergroup: should read bitnami:supergroup.
>>> How can I change this?
>>>
>>>
>>
>>
>> --
>> Bertrand Dechoux
>>
>
>
--
Bertrand Dechoux
Re: Permission denied hadoop job
Posted by Bertrand Dechoux <de...@gmail.com>.
It wasn't meant to be a sardonic retort and I apologize if it did sound
like it.
A few times, people ask questions which make think that they didn't have
access to the right documentation.
I believe I provided you enough information about your specific issue and a
way to understand better what you did right or wrong.
About security, without Kerberos there is no strong authentification and as
Hadoop does not warn (as I far as I know) against the superuser being the
system root, a few times people may believe there isn't any security and
thus may not understand the error thrown by the system. Which is why
knowing about any official documentation on the subject is important.
Regards
Bertrand
On Sun, Aug 19, 2012 at 8:34 AM, Robert Irribarren <ro...@algorithms.io>wrote:
> Thanks for the sardonic retort, I got it working. I just had to redo the
> steps and I set the hdfs to my tmp folder and had permissions set up
> properly
>
>
> On Sat, Aug 18, 2012 at 8:50 PM, Bertrand Dechoux <de...@gmail.com>wrote:
>
>> Hi,
>>
>> I have never heard about a magic group called 'hadoop'.
>> Does it have a specific meaning in your context?
>>
>> You should read more about security in Hadoop.
>> Your last question hint that you haven't looked at it.
>>
>>
>> http://hadoop.apache.org/common/docs/r1.0.3/hdfs_user_guide.html#File+Permissions+and+Security
>>
>> File Permissions and Security
>>> The file permissions are designed to be similar to file permissions on
>>> other familiar platforms like Linux. Currently, security is limited to
>>> simple file permissions. *The user that starts NameNode is treated as
>>> the superuser for HDFS.* Future versions of HDFS will support network
>>> authentication protocols like Kerberos for user authentication and
>>> encryption of data transfers. The details are discussed in the Permissions
>>> Guide<http://hadoop.apache.org/common/docs/r1.0.3/hdfs_permissions_guide.html>
>>> .
>>
>>
>> The straight answer is : run your cluster with the bitnami user.
>>
>> But it may be a good habit to
>> 1) have the hadoop superuser being different than the system root
>> 2) interact with your cluster with users having limited access ie not
>> your superuser
>>
>> Regards
>>
>> Bertrand
>>
>> On Sun, Aug 19, 2012 at 3:43 AM, Robert Irribarren <ro...@algorithms.io>wrote:
>>
>>> bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>>> -copyFromLocal /tmp/myDir /user/bitnami/myDir
>>> Warning: $HADOOP_HOME is deprecated.
>>>
>>> copyFromLocal: Target /user/bitnami/myDir/ is a directory
>>> bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>>> -rmr /user/bitnami/
>>> Warning: $HADOOP_HOME is deprecated.
>>>
>>> rmr: org.apache.hadoop.security.AccessControlException: Permission
>>> denied: user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
>>>
>>>
>>> I have also made sure that bitnami was in the group hadoop
>>>
>>> bitnami@ip-10-xxx-xx-xx:groups bitnami
>>> bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev
>>> netdev admin hadoop
>>>
>>> I have read that the root:supergroup: should read bitnami:supergroup.
>>> How can I change this?
>>>
>>>
>>
>>
>> --
>> Bertrand Dechoux
>>
>
>
--
Bertrand Dechoux
Re: Permission denied hadoop job
Posted by Bertrand Dechoux <de...@gmail.com>.
It wasn't meant to be a sardonic retort and I apologize if it did sound
like it.
A few times, people ask questions which make think that they didn't have
access to the right documentation.
I believe I provided you enough information about your specific issue and a
way to understand better what you did right or wrong.
About security, without Kerberos there is no strong authentification and as
Hadoop does not warn (as I far as I know) against the superuser being the
system root, a few times people may believe there isn't any security and
thus may not understand the error thrown by the system. Which is why
knowing about any official documentation on the subject is important.
Regards
Bertrand
On Sun, Aug 19, 2012 at 8:34 AM, Robert Irribarren <ro...@algorithms.io>wrote:
> Thanks for the sardonic retort, I got it working. I just had to redo the
> steps and I set the hdfs to my tmp folder and had permissions set up
> properly
>
>
> On Sat, Aug 18, 2012 at 8:50 PM, Bertrand Dechoux <de...@gmail.com>wrote:
>
>> Hi,
>>
>> I have never heard about a magic group called 'hadoop'.
>> Does it have a specific meaning in your context?
>>
>> You should read more about security in Hadoop.
>> Your last question hint that you haven't looked at it.
>>
>>
>> http://hadoop.apache.org/common/docs/r1.0.3/hdfs_user_guide.html#File+Permissions+and+Security
>>
>> File Permissions and Security
>>> The file permissions are designed to be similar to file permissions on
>>> other familiar platforms like Linux. Currently, security is limited to
>>> simple file permissions. *The user that starts NameNode is treated as
>>> the superuser for HDFS.* Future versions of HDFS will support network
>>> authentication protocols like Kerberos for user authentication and
>>> encryption of data transfers. The details are discussed in the Permissions
>>> Guide<http://hadoop.apache.org/common/docs/r1.0.3/hdfs_permissions_guide.html>
>>> .
>>
>>
>> The straight answer is : run your cluster with the bitnami user.
>>
>> But it may be a good habit to
>> 1) have the hadoop superuser being different than the system root
>> 2) interact with your cluster with users having limited access ie not
>> your superuser
>>
>> Regards
>>
>> Bertrand
>>
>> On Sun, Aug 19, 2012 at 3:43 AM, Robert Irribarren <ro...@algorithms.io>wrote:
>>
>>> bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>>> -copyFromLocal /tmp/myDir /user/bitnami/myDir
>>> Warning: $HADOOP_HOME is deprecated.
>>>
>>> copyFromLocal: Target /user/bitnami/myDir/ is a directory
>>> bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>>> -rmr /user/bitnami/
>>> Warning: $HADOOP_HOME is deprecated.
>>>
>>> rmr: org.apache.hadoop.security.AccessControlException: Permission
>>> denied: user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
>>>
>>>
>>> I have also made sure that bitnami was in the group hadoop
>>>
>>> bitnami@ip-10-xxx-xx-xx:groups bitnami
>>> bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev
>>> netdev admin hadoop
>>>
>>> I have read that the root:supergroup: should read bitnami:supergroup.
>>> How can I change this?
>>>
>>>
>>
>>
>> --
>> Bertrand Dechoux
>>
>
>
--
Bertrand Dechoux
Re: Permission denied hadoop job
Posted by Bertrand Dechoux <de...@gmail.com>.
It wasn't meant to be a sardonic retort and I apologize if it did sound
like it.
A few times, people ask questions which make think that they didn't have
access to the right documentation.
I believe I provided you enough information about your specific issue and a
way to understand better what you did right or wrong.
About security, without Kerberos there is no strong authentification and as
Hadoop does not warn (as I far as I know) against the superuser being the
system root, a few times people may believe there isn't any security and
thus may not understand the error thrown by the system. Which is why
knowing about any official documentation on the subject is important.
Regards
Bertrand
On Sun, Aug 19, 2012 at 8:34 AM, Robert Irribarren <ro...@algorithms.io>wrote:
> Thanks for the sardonic retort, I got it working. I just had to redo the
> steps and I set the hdfs to my tmp folder and had permissions set up
> properly
>
>
> On Sat, Aug 18, 2012 at 8:50 PM, Bertrand Dechoux <de...@gmail.com>wrote:
>
>> Hi,
>>
>> I have never heard about a magic group called 'hadoop'.
>> Does it have a specific meaning in your context?
>>
>> You should read more about security in Hadoop.
>> Your last question hint that you haven't looked at it.
>>
>>
>> http://hadoop.apache.org/common/docs/r1.0.3/hdfs_user_guide.html#File+Permissions+and+Security
>>
>> File Permissions and Security
>>> The file permissions are designed to be similar to file permissions on
>>> other familiar platforms like Linux. Currently, security is limited to
>>> simple file permissions. *The user that starts NameNode is treated as
>>> the superuser for HDFS.* Future versions of HDFS will support network
>>> authentication protocols like Kerberos for user authentication and
>>> encryption of data transfers. The details are discussed in the Permissions
>>> Guide<http://hadoop.apache.org/common/docs/r1.0.3/hdfs_permissions_guide.html>
>>> .
>>
>>
>> The straight answer is : run your cluster with the bitnami user.
>>
>> But it may be a good habit to
>> 1) have the hadoop superuser being different than the system root
>> 2) interact with your cluster with users having limited access ie not
>> your superuser
>>
>> Regards
>>
>> Bertrand
>>
>> On Sun, Aug 19, 2012 at 3:43 AM, Robert Irribarren <ro...@algorithms.io>wrote:
>>
>>> bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>>> -copyFromLocal /tmp/myDir /user/bitnami/myDir
>>> Warning: $HADOOP_HOME is deprecated.
>>>
>>> copyFromLocal: Target /user/bitnami/myDir/ is a directory
>>> bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>>> -rmr /user/bitnami/
>>> Warning: $HADOOP_HOME is deprecated.
>>>
>>> rmr: org.apache.hadoop.security.AccessControlException: Permission
>>> denied: user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
>>>
>>>
>>> I have also made sure that bitnami was in the group hadoop
>>>
>>> bitnami@ip-10-xxx-xx-xx:groups bitnami
>>> bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev
>>> netdev admin hadoop
>>>
>>> I have read that the root:supergroup: should read bitnami:supergroup.
>>> How can I change this?
>>>
>>>
>>
>>
>> --
>> Bertrand Dechoux
>>
>
>
--
Bertrand Dechoux
Re: Permission denied hadoop job
Posted by Robert Irribarren <ro...@algorithms.io>.
Thanks for the sardonic retort, I got it working. I just had to redo the
steps and I set the hdfs to my tmp folder and had permissions set up
properly
On Sat, Aug 18, 2012 at 8:50 PM, Bertrand Dechoux <de...@gmail.com>wrote:
> Hi,
>
> I have never heard about a magic group called 'hadoop'.
> Does it have a specific meaning in your context?
>
> You should read more about security in Hadoop.
> Your last question hint that you haven't looked at it.
>
>
> http://hadoop.apache.org/common/docs/r1.0.3/hdfs_user_guide.html#File+Permissions+and+Security
>
> File Permissions and Security
>> The file permissions are designed to be similar to file permissions on
>> other familiar platforms like Linux. Currently, security is limited to
>> simple file permissions. *The user that starts NameNode is treated as
>> the superuser for HDFS.* Future versions of HDFS will support network
>> authentication protocols like Kerberos for user authentication and
>> encryption of data transfers. The details are discussed in the Permissions
>> Guide<http://hadoop.apache.org/common/docs/r1.0.3/hdfs_permissions_guide.html>
>> .
>
>
> The straight answer is : run your cluster with the bitnami user.
>
> But it may be a good habit to
> 1) have the hadoop superuser being different than the system root
> 2) interact with your cluster with users having limited access ie not your
> superuser
>
> Regards
>
> Bertrand
>
> On Sun, Aug 19, 2012 at 3:43 AM, Robert Irribarren <ro...@algorithms.io>wrote:
>
>> bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>> -copyFromLocal /tmp/myDir /user/bitnami/myDir
>> Warning: $HADOOP_HOME is deprecated.
>>
>> copyFromLocal: Target /user/bitnami/myDir/ is a directory
>> bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>> -rmr /user/bitnami/
>> Warning: $HADOOP_HOME is deprecated.
>>
>> rmr: org.apache.hadoop.security.AccessControlException: Permission
>> denied: user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
>>
>>
>> I have also made sure that bitnami was in the group hadoop
>>
>> bitnami@ip-10-xxx-xx-xx:groups bitnami
>> bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev netdev
>> admin hadoop
>>
>> I have read that the root:supergroup: should read bitnami:supergroup. How
>> can I change this?
>>
>>
>
>
> --
> Bertrand Dechoux
>
Re: Permission denied hadoop job
Posted by Robert Irribarren <ro...@algorithms.io>.
Thanks for the sardonic retort, I got it working. I just had to redo the
steps and I set the hdfs to my tmp folder and had permissions set up
properly
On Sat, Aug 18, 2012 at 8:50 PM, Bertrand Dechoux <de...@gmail.com>wrote:
> Hi,
>
> I have never heard about a magic group called 'hadoop'.
> Does it have a specific meaning in your context?
>
> You should read more about security in Hadoop.
> Your last question hint that you haven't looked at it.
>
>
> http://hadoop.apache.org/common/docs/r1.0.3/hdfs_user_guide.html#File+Permissions+and+Security
>
> File Permissions and Security
>> The file permissions are designed to be similar to file permissions on
>> other familiar platforms like Linux. Currently, security is limited to
>> simple file permissions. *The user that starts NameNode is treated as
>> the superuser for HDFS.* Future versions of HDFS will support network
>> authentication protocols like Kerberos for user authentication and
>> encryption of data transfers. The details are discussed in the Permissions
>> Guide<http://hadoop.apache.org/common/docs/r1.0.3/hdfs_permissions_guide.html>
>> .
>
>
> The straight answer is : run your cluster with the bitnami user.
>
> But it may be a good habit to
> 1) have the hadoop superuser being different than the system root
> 2) interact with your cluster with users having limited access ie not your
> superuser
>
> Regards
>
> Bertrand
>
> On Sun, Aug 19, 2012 at 3:43 AM, Robert Irribarren <ro...@algorithms.io>wrote:
>
>> bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>> -copyFromLocal /tmp/myDir /user/bitnami/myDir
>> Warning: $HADOOP_HOME is deprecated.
>>
>> copyFromLocal: Target /user/bitnami/myDir/ is a directory
>> bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>> -rmr /user/bitnami/
>> Warning: $HADOOP_HOME is deprecated.
>>
>> rmr: org.apache.hadoop.security.AccessControlException: Permission
>> denied: user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
>>
>>
>> I have also made sure that bitnami was in the group hadoop
>>
>> bitnami@ip-10-xxx-xx-xx:groups bitnami
>> bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev netdev
>> admin hadoop
>>
>> I have read that the root:supergroup: should read bitnami:supergroup. How
>> can I change this?
>>
>>
>
>
> --
> Bertrand Dechoux
>
Re: Permission denied hadoop job
Posted by Robert Irribarren <ro...@algorithms.io>.
Thanks for the sardonic retort, I got it working. I just had to redo the
steps and I set the hdfs to my tmp folder and had permissions set up
properly
On Sat, Aug 18, 2012 at 8:50 PM, Bertrand Dechoux <de...@gmail.com>wrote:
> Hi,
>
> I have never heard about a magic group called 'hadoop'.
> Does it have a specific meaning in your context?
>
> You should read more about security in Hadoop.
> Your last question hint that you haven't looked at it.
>
>
> http://hadoop.apache.org/common/docs/r1.0.3/hdfs_user_guide.html#File+Permissions+and+Security
>
> File Permissions and Security
>> The file permissions are designed to be similar to file permissions on
>> other familiar platforms like Linux. Currently, security is limited to
>> simple file permissions. *The user that starts NameNode is treated as
>> the superuser for HDFS.* Future versions of HDFS will support network
>> authentication protocols like Kerberos for user authentication and
>> encryption of data transfers. The details are discussed in the Permissions
>> Guide<http://hadoop.apache.org/common/docs/r1.0.3/hdfs_permissions_guide.html>
>> .
>
>
> The straight answer is : run your cluster with the bitnami user.
>
> But it may be a good habit to
> 1) have the hadoop superuser being different than the system root
> 2) interact with your cluster with users having limited access ie not your
> superuser
>
> Regards
>
> Bertrand
>
> On Sun, Aug 19, 2012 at 3:43 AM, Robert Irribarren <ro...@algorithms.io>wrote:
>
>> bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>> -copyFromLocal /tmp/myDir /user/bitnami/myDir
>> Warning: $HADOOP_HOME is deprecated.
>>
>> copyFromLocal: Target /user/bitnami/myDir/ is a directory
>> bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>> -rmr /user/bitnami/
>> Warning: $HADOOP_HOME is deprecated.
>>
>> rmr: org.apache.hadoop.security.AccessControlException: Permission
>> denied: user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
>>
>>
>> I have also made sure that bitnami was in the group hadoop
>>
>> bitnami@ip-10-xxx-xx-xx:groups bitnami
>> bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev netdev
>> admin hadoop
>>
>> I have read that the root:supergroup: should read bitnami:supergroup. How
>> can I change this?
>>
>>
>
>
> --
> Bertrand Dechoux
>
Re: Permission denied hadoop job
Posted by Robert Irribarren <ro...@algorithms.io>.
Thanks for the sardonic retort, I got it working. I just had to redo the
steps and I set the hdfs to my tmp folder and had permissions set up
properly
On Sat, Aug 18, 2012 at 8:50 PM, Bertrand Dechoux <de...@gmail.com>wrote:
> Hi,
>
> I have never heard about a magic group called 'hadoop'.
> Does it have a specific meaning in your context?
>
> You should read more about security in Hadoop.
> Your last question hint that you haven't looked at it.
>
>
> http://hadoop.apache.org/common/docs/r1.0.3/hdfs_user_guide.html#File+Permissions+and+Security
>
> File Permissions and Security
>> The file permissions are designed to be similar to file permissions on
>> other familiar platforms like Linux. Currently, security is limited to
>> simple file permissions. *The user that starts NameNode is treated as
>> the superuser for HDFS.* Future versions of HDFS will support network
>> authentication protocols like Kerberos for user authentication and
>> encryption of data transfers. The details are discussed in the Permissions
>> Guide<http://hadoop.apache.org/common/docs/r1.0.3/hdfs_permissions_guide.html>
>> .
>
>
> The straight answer is : run your cluster with the bitnami user.
>
> But it may be a good habit to
> 1) have the hadoop superuser being different than the system root
> 2) interact with your cluster with users having limited access ie not your
> superuser
>
> Regards
>
> Bertrand
>
> On Sun, Aug 19, 2012 at 3:43 AM, Robert Irribarren <ro...@algorithms.io>wrote:
>
>> bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>> -copyFromLocal /tmp/myDir /user/bitnami/myDir
>> Warning: $HADOOP_HOME is deprecated.
>>
>> copyFromLocal: Target /user/bitnami/myDir/ is a directory
>> bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
>> -rmr /user/bitnami/
>> Warning: $HADOOP_HOME is deprecated.
>>
>> rmr: org.apache.hadoop.security.AccessControlException: Permission
>> denied: user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
>>
>>
>> I have also made sure that bitnami was in the group hadoop
>>
>> bitnami@ip-10-xxx-xx-xx:groups bitnami
>> bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev netdev
>> admin hadoop
>>
>> I have read that the root:supergroup: should read bitnami:supergroup. How
>> can I change this?
>>
>>
>
>
> --
> Bertrand Dechoux
>
Re: Permission denied hadoop job
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
I have never heard about a magic group called 'hadoop'.
Does it have a specific meaning in your context?
You should read more about security in Hadoop.
Your last question hint that you haven't looked at it.
http://hadoop.apache.org/common/docs/r1.0.3/hdfs_user_guide.html#File+Permissions+and+Security
File Permissions and Security
> The file permissions are designed to be similar to file permissions on
> other familiar platforms like Linux. Currently, security is limited to
> simple file permissions. *The user that starts NameNode is treated as the
> superuser for HDFS.* Future versions of HDFS will support network
> authentication protocols like Kerberos for user authentication and
> encryption of data transfers. The details are discussed in the Permissions
> Guide<http://hadoop.apache.org/common/docs/r1.0.3/hdfs_permissions_guide.html>
> .
The straight answer is : run your cluster with the bitnami user.
But it may be a good habit to
1) have the hadoop superuser being different than the system root
2) interact with your cluster with users having limited access ie not your
superuser
Regards
Bertrand
On Sun, Aug 19, 2012 at 3:43 AM, Robert Irribarren <ro...@algorithms.io>wrote:
> bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
> -copyFromLocal /tmp/myDir /user/bitnami/myDir
> Warning: $HADOOP_HOME is deprecated.
>
> copyFromLocal: Target /user/bitnami/myDir/ is a directory
> bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
> -rmr /user/bitnami/
> Warning: $HADOOP_HOME is deprecated.
>
> rmr: org.apache.hadoop.security.AccessControlException: Permission denied:
> user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
>
>
> I have also made sure that bitnami was in the group hadoop
>
> bitnami@ip-10-xxx-xx-xx:groups bitnami
> bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev netdev
> admin hadoop
>
> I have read that the root:supergroup: should read bitnami:supergroup. How
> can I change this?
>
>
--
Bertrand Dechoux
Re: Permission denied hadoop job
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
I have never heard about a magic group called 'hadoop'.
Does it have a specific meaning in your context?
You should read more about security in Hadoop.
Your last question hint that you haven't looked at it.
http://hadoop.apache.org/common/docs/r1.0.3/hdfs_user_guide.html#File+Permissions+and+Security
File Permissions and Security
> The file permissions are designed to be similar to file permissions on
> other familiar platforms like Linux. Currently, security is limited to
> simple file permissions. *The user that starts NameNode is treated as the
> superuser for HDFS.* Future versions of HDFS will support network
> authentication protocols like Kerberos for user authentication and
> encryption of data transfers. The details are discussed in the Permissions
> Guide<http://hadoop.apache.org/common/docs/r1.0.3/hdfs_permissions_guide.html>
> .
The straight answer is : run your cluster with the bitnami user.
But it may be a good habit to
1) have the hadoop superuser being different than the system root
2) interact with your cluster with users having limited access ie not your
superuser
Regards
Bertrand
On Sun, Aug 19, 2012 at 3:43 AM, Robert Irribarren <ro...@algorithms.io>wrote:
> bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
> -copyFromLocal /tmp/myDir /user/bitnami/myDir
> Warning: $HADOOP_HOME is deprecated.
>
> copyFromLocal: Target /user/bitnami/myDir/ is a directory
> bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
> -rmr /user/bitnami/
> Warning: $HADOOP_HOME is deprecated.
>
> rmr: org.apache.hadoop.security.AccessControlException: Permission denied:
> user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
>
>
> I have also made sure that bitnami was in the group hadoop
>
> bitnami@ip-10-xxx-xx-xx:groups bitnami
> bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev netdev
> admin hadoop
>
> I have read that the root:supergroup: should read bitnami:supergroup. How
> can I change this?
>
>
--
Bertrand Dechoux
Re: Permission denied hadoop job
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
I have never heard about a magic group called 'hadoop'.
Does it have a specific meaning in your context?
You should read more about security in Hadoop.
Your last question hint that you haven't looked at it.
http://hadoop.apache.org/common/docs/r1.0.3/hdfs_user_guide.html#File+Permissions+and+Security
File Permissions and Security
> The file permissions are designed to be similar to file permissions on
> other familiar platforms like Linux. Currently, security is limited to
> simple file permissions. *The user that starts NameNode is treated as the
> superuser for HDFS.* Future versions of HDFS will support network
> authentication protocols like Kerberos for user authentication and
> encryption of data transfers. The details are discussed in the Permissions
> Guide<http://hadoop.apache.org/common/docs/r1.0.3/hdfs_permissions_guide.html>
> .
The straight answer is : run your cluster with the bitnami user.
But it may be a good habit to
1) have the hadoop superuser being different than the system root
2) interact with your cluster with users having limited access ie not your
superuser
Regards
Bertrand
On Sun, Aug 19, 2012 at 3:43 AM, Robert Irribarren <ro...@algorithms.io>wrote:
> bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
> -copyFromLocal /tmp/myDir /user/bitnami/myDir
> Warning: $HADOOP_HOME is deprecated.
>
> copyFromLocal: Target /user/bitnami/myDir/ is a directory
> bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
> -rmr /user/bitnami/
> Warning: $HADOOP_HOME is deprecated.
>
> rmr: org.apache.hadoop.security.AccessControlException: Permission denied:
> user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
>
>
> I have also made sure that bitnami was in the group hadoop
>
> bitnami@ip-10-xxx-xx-xx:groups bitnami
> bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev netdev
> admin hadoop
>
> I have read that the root:supergroup: should read bitnami:supergroup. How
> can I change this?
>
>
--
Bertrand Dechoux
Re: Permission denied hadoop job
Posted by Bertrand Dechoux <de...@gmail.com>.
Hi,
I have never heard about a magic group called 'hadoop'.
Does it have a specific meaning in your context?
You should read more about security in Hadoop.
Your last question hint that you haven't looked at it.
http://hadoop.apache.org/common/docs/r1.0.3/hdfs_user_guide.html#File+Permissions+and+Security
File Permissions and Security
> The file permissions are designed to be similar to file permissions on
> other familiar platforms like Linux. Currently, security is limited to
> simple file permissions. *The user that starts NameNode is treated as the
> superuser for HDFS.* Future versions of HDFS will support network
> authentication protocols like Kerberos for user authentication and
> encryption of data transfers. The details are discussed in the Permissions
> Guide<http://hadoop.apache.org/common/docs/r1.0.3/hdfs_permissions_guide.html>
> .
The straight answer is : run your cluster with the bitnami user.
But it may be a good habit to
1) have the hadoop superuser being different than the system root
2) interact with your cluster with users having limited access ie not your
superuser
Regards
Bertrand
On Sun, Aug 19, 2012 at 3:43 AM, Robert Irribarren <ro...@algorithms.io>wrote:
> bitnami@ip-10-252-38-45:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
> -copyFromLocal /tmp/myDir /user/bitnami/myDir
> Warning: $HADOOP_HOME is deprecated.
>
> copyFromLocal: Target /user/bitnami/myDir/ is a directory
> bitnami@ip-10-xxx-xx-xx:/usr/share/hadoop/hadoop-1.0.3$ bin/hadoop dfs
> -rmr /user/bitnami/
> Warning: $HADOOP_HOME is deprecated.
>
> rmr: org.apache.hadoop.security.AccessControlException: Permission denied:
> user=bitnami, access=WRITE, inode="user":root:supergroup:rwxr-xr-x
>
>
> I have also made sure that bitnami was in the group hadoop
>
> bitnami@ip-10-xxx-xx-xx:groups bitnami
> bitnami : bitnami adm dialout cdrom floppy audio dip video plugdev netdev
> admin hadoop
>
> I have read that the root:supergroup: should read bitnami:supergroup. How
> can I change this?
>
>
--
Bertrand Dechoux