You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Suman Sehgal (JIRA)" <ji...@apache.org> on 2008/08/07 06:57:44 UTC
[jira] Created: (HADOOP-3913) Distcp allows copy even if directory
permissions are 000.
Distcp allows copy even if directory permissions are 000.
---------------------------------------------------------
Key: HADOOP-3913
URL: https://issues.apache.org/jira/browse/HADOOP-3913
Project: Hadoop Core
Issue Type: Bug
Components: tools/distcp
Affects Versions: 0.18.0
Reporter: Suman Sehgal
distcp initiated from 0.17.0 and 0.18.0 permissions turned ON, directory with permission 000, data gets copied from
0.18.0 to 0.17.0
Data gets copied between different versions of hadoop when tried by the same user. It gets failed only if other
user(other than the user who started cluster) tries to copy data using distcp.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (HADOOP-3913) Distcp allows copy even
if directory permissions are 000.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623468#action_12623468 ]
szetszwo edited comment on HADOOP-3913 at 8/18/08 2:34 PM:
-------------------------------------------------------------------------
Just want to clarify that a non-superuser is not allowed to access a directory with 000 permissions in HDFS, even the user is the owner of the directory.
Documentation about this can be found in the "The Super-User" section in "Permissions User and Administrator Guide", http://hadoop.apache.org/core/docs/current/hdfs_permissions_guide.html#The+Super-User
was (Author: szetszwo):
Just want to clarify that if a non-superuser is not allowed to access a directory with 000 permissions in HDFS, even the user is the owner of the directory.
Documentation about this can be found in the "The Super-User" section in "Permissions User and Administrator Guide", http://hadoop.apache.org/core/docs/current/hdfs_permissions_guide.html#The+Super-User
> Distcp allows copy even if directory permissions are 000.
> ---------------------------------------------------------
>
> Key: HADOOP-3913
> URL: https://issues.apache.org/jira/browse/HADOOP-3913
> Project: Hadoop Core
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 0.18.0
> Reporter: Suman Sehgal
>
> distcp initiated from 0.17.0 and 0.18.0 permissions turned ON, directory with permission 000, data gets copied from
> 0.18.0 to 0.17.0
> Data gets copied between different versions of hadoop when tried by the same user. It gets failed only if other
> user(other than the user who started cluster) tries to copy data using distcp.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HADOOP-3913) Distcp allows copy even if directory
permissions are 000.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tsz Wo (Nicholas), SZE resolved HADOOP-3913.
--------------------------------------------
Resolution: Invalid
Closing this as invalid.
> Distcp allows copy even if directory permissions are 000.
> ---------------------------------------------------------
>
> Key: HADOOP-3913
> URL: https://issues.apache.org/jira/browse/HADOOP-3913
> Project: Hadoop Core
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 0.18.0
> Reporter: Suman Sehgal
>
> distcp initiated from 0.17.0 and 0.18.0 permissions turned ON, directory with permission 000, data gets copied from
> 0.18.0 to 0.17.0
> Data gets copied between different versions of hadoop when tried by the same user. It gets failed only if other
> user(other than the user who started cluster) tries to copy data using distcp.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3913) Distcp allows copy even if
directory permissions are 000.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623468#action_12623468 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3913:
------------------------------------------------
Just want to clarify that if a non-superuser is not allowed to access a directory with 000 permissions in HDFS, even the user is the owner of the directory.
Documentation about this can be found in the "The Super-User" section in "Permissions User and Administrator Guide", http://hadoop.apache.org/core/docs/current/hdfs_permissions_guide.html#The+Super-User
> Distcp allows copy even if directory permissions are 000.
> ---------------------------------------------------------
>
> Key: HADOOP-3913
> URL: https://issues.apache.org/jira/browse/HADOOP-3913
> Project: Hadoop Core
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 0.18.0
> Reporter: Suman Sehgal
>
> distcp initiated from 0.17.0 and 0.18.0 permissions turned ON, directory with permission 000, data gets copied from
> 0.18.0 to 0.17.0
> Data gets copied between different versions of hadoop when tried by the same user. It gets failed only if other
> user(other than the user who started cluster) tries to copy data using distcp.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3913) Distcp allows copy even if
directory permissions are 000.
Posted by "Suman Sehgal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12621221#action_12621221 ]
Suman Sehgal commented on HADOOP-3913:
--------------------------------------
User mentioned here is not a superuser but group is "supergroup". Tried above mentioned scenario after changing the group of the user but copying is still possible.
> Distcp allows copy even if directory permissions are 000.
> ---------------------------------------------------------
>
> Key: HADOOP-3913
> URL: https://issues.apache.org/jira/browse/HADOOP-3913
> Project: Hadoop Core
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 0.18.0
> Reporter: Suman Sehgal
>
> distcp initiated from 0.17.0 and 0.18.0 permissions turned ON, directory with permission 000, data gets copied from
> 0.18.0 to 0.17.0
> Data gets copied between different versions of hadoop when tried by the same user. It gets failed only if other
> user(other than the user who started cluster) tries to copy data using distcp.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3913) Distcp allows copy even if
directory permissions are 000.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622340#action_12622340 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3913:
------------------------------------------------
I tried the following
# Login as tsz
## created a 0.17 cluster
## created a 0.18 cluster
## created home directory for "sze" in both cluster
# Login as sze (a non-superuser for both clusters)
## mkdir and copied some files to /user/sze/foo in the 0.18 cluster
## distcp the files in /user/sze/foo from the 0.18 cluster to the 0.17 cluster (success)
## chmod 000 /user/sze/foo in the 0.18 cluster
## distcp the files in /user/sze/foo from the 0.18 cluster to the 0.17 cluster (failed with the same error message shown in HADOOP-3889)
So, I cannot reproduce it. Could you check it, Suman?
> Distcp allows copy even if directory permissions are 000.
> ---------------------------------------------------------
>
> Key: HADOOP-3913
> URL: https://issues.apache.org/jira/browse/HADOOP-3913
> Project: Hadoop Core
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 0.18.0
> Reporter: Suman Sehgal
>
> distcp initiated from 0.17.0 and 0.18.0 permissions turned ON, directory with permission 000, data gets copied from
> 0.18.0 to 0.17.0
> Data gets copied between different versions of hadoop when tried by the same user. It gets failed only if other
> user(other than the user who started cluster) tries to copy data using distcp.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3913) Distcp allows copy even if
directory permissions are 000.
Posted by "Suman Sehgal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12623306#action_12623306 ]
Suman Sehgal commented on HADOOP-3913:
--------------------------------------
Yeah the distcp operation was carried out by the same user who created directory (as mentioned by you in previous comment). Superuser was able to access the directory with 000 permissions but in normal Linux enviornment a user who creates the directory, can't access its contents on 000 permission. If its the special scenario of HDFS then it should be documented.
> Distcp allows copy even if directory permissions are 000.
> ---------------------------------------------------------
>
> Key: HADOOP-3913
> URL: https://issues.apache.org/jira/browse/HADOOP-3913
> Project: Hadoop Core
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 0.18.0
> Reporter: Suman Sehgal
>
> distcp initiated from 0.17.0 and 0.18.0 permissions turned ON, directory with permission 000, data gets copied from
> 0.18.0 to 0.17.0
> Data gets copied between different versions of hadoop when tried by the same user. It gets failed only if other
> user(other than the user who started cluster) tries to copy data using distcp.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3913) Distcp allows copy even if
directory permissions are 000.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12621512#action_12621512 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3913:
------------------------------------------------
By superusers (in hadoop), I mean the user started the namenode or the users belonging one of the groups specified in dfs.permissions.supergroup. In your case, you are copying from a 0.18 cluster to a 0.17 cluster. Note that your 0.18 superusers and 0.17 superusers could be different.
I guess you might have used a superuser account in your 0.18 cluster to do distcp. So the copy success even the directory is 000. I will check this.
> Distcp allows copy even if directory permissions are 000.
> ---------------------------------------------------------
>
> Key: HADOOP-3913
> URL: https://issues.apache.org/jira/browse/HADOOP-3913
> Project: Hadoop Core
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 0.18.0
> Reporter: Suman Sehgal
>
> distcp initiated from 0.17.0 and 0.18.0 permissions turned ON, directory with permission 000, data gets copied from
> 0.18.0 to 0.17.0
> Data gets copied between different versions of hadoop when tried by the same user. It gets failed only if other
> user(other than the user who started cluster) tries to copy data using distcp.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3913) Distcp allows copy even if
directory permissions are 000.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12622644#action_12622644 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3913:
------------------------------------------------
I have submitted a patch in HADOOP-3889, which fixes the error message problem.
There is a new test in the patch, such that it starts up a cluster with one user account and run distcp with another user account. It makes sure that distcp will fail when the source directory permission is 000.
> Distcp allows copy even if directory permissions are 000.
> ---------------------------------------------------------
>
> Key: HADOOP-3913
> URL: https://issues.apache.org/jira/browse/HADOOP-3913
> Project: Hadoop Core
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 0.18.0
> Reporter: Suman Sehgal
>
> distcp initiated from 0.17.0 and 0.18.0 permissions turned ON, directory with permission 000, data gets copied from
> 0.18.0 to 0.17.0
> Data gets copied between different versions of hadoop when tried by the same user. It gets failed only if other
> user(other than the user who started cluster) tries to copy data using distcp.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3913) Distcp allows copy even if
directory permissions are 000.
Posted by "Suman Sehgal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620510#action_12620510 ]
Suman Sehgal commented on HADOOP-3913:
--------------------------------------
When different user tries to copy data between different versions of hadoop for the above mentioned scenario then
following error message appears:
bash-3.00$ hadoop distcp -conf <conf-dir>/hadoop-site.xml -p hftp://<nn1>:<port>/docs/linkmap.html
hdfs://<nn2>:<port>/suman
08/08/01 10:52:11 INFO util.CopyFiles: srcPaths=[hftp://<nn1>:<port>/docs/linkmap.html]
08/08/01 10:52:11 INFO util.CopyFiles: destPath=hdfs://<nn2>:<port>/suman
-------------------------------------------------------------------------------
With failures, global counters are inaccurate; consider running with -i
Copy failed: org.apache.hadoop.mapred.InvalidInputException: Input source
hftp://<nn1>:<port>/docs/linkmap.html does not exist.
--------------------------------------------------------------------------------
at org.apache.hadoop.util.CopyFiles.checkSrcPath(CopyFiles.java:578)
at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:594)
at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
Instead of the above error message, "Permission denied" message should be displayed in order to aviod
confusion.
> Distcp allows copy even if directory permissions are 000.
> ---------------------------------------------------------
>
> Key: HADOOP-3913
> URL: https://issues.apache.org/jira/browse/HADOOP-3913
> Project: Hadoop Core
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 0.18.0
> Reporter: Suman Sehgal
>
> distcp initiated from 0.17.0 and 0.18.0 permissions turned ON, directory with permission 000, data gets copied from
> 0.18.0 to 0.17.0
> Data gets copied between different versions of hadoop when tried by the same user. It gets failed only if other
> user(other than the user who started cluster) tries to copy data using distcp.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-3913) Distcp allows copy even if
directory permissions are 000.
Posted by "Tsz Wo (Nicholas), SZE (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-3913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620692#action_12620692 ]
Tsz Wo (Nicholas), SZE commented on HADOOP-3913:
------------------------------------------------
> Instead of the above error message, "Permission denied" message should be displayed in order to aviod confusion.
+1 the error message is not clear
BTW, are you sure that you are not a superuser when you copy files from a directory with permission 000? All permission checking will be ignored for superusers.
> Distcp allows copy even if directory permissions are 000.
> ---------------------------------------------------------
>
> Key: HADOOP-3913
> URL: https://issues.apache.org/jira/browse/HADOOP-3913
> Project: Hadoop Core
> Issue Type: Bug
> Components: tools/distcp
> Affects Versions: 0.18.0
> Reporter: Suman Sehgal
>
> distcp initiated from 0.17.0 and 0.18.0 permissions turned ON, directory with permission 000, data gets copied from
> 0.18.0 to 0.17.0
> Data gets copied between different versions of hadoop when tried by the same user. It gets failed only if other
> user(other than the user who started cluster) tries to copy data using distcp.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.